Azure Bits #3 – Adding a Message to an Azure Queue

In Azure Bits #2 – Saving the Image to Azure Blob Storage, we were able to save our image to Azure Blob Storage and verify that this all went according to plan after re-publishing our site to Azure.

In this Azure Bit, we will take a look at the Azure Queue service and we will place a message in the queue to signal that our newly-uploaded image is ready to be processed.

The first thing we will need to do is to create our IQueueService/QueueService to abstract the interaction with the Azure Queue service.  We’ll just need one method for now.

 

IQueueService.cs
  1. public interface IQueueService<T> where T: new()
  2. {
  3.     Task AddMessageToQueueAsync(string messageId, T messageObject);
  4. }

We’ll need to know the queue name for our Azure Queue and we’ll need the Blob Storage Connection string, so I’ll go ahead and add the ImagesQueue name to our appSettings in web.config.

web.config – appSettings
  1. <appSettings>
  2.   <add key=ImageRootPath value=https://imagemanipulator3.blob.core.windows.net/images />
  3.   <add key=ImagesContainer value=images />
  4.   <add key=ImagesQueue value=images />
  5. </appSettings>

 

Finally, here’s the initial skeleton of QueueService.  You’ll  see that I went ahead and fetched the connection string and queue name in the constructor of QueueService.

 

QueueService.cs
  1. public class QueueService<T> : IQueueService<T> where T : class, new()
  2. {
  3.     private readonly string _connectionString;
  4.     private readonly string _queueName;
  5.     public QueueService()
  6.     {
  7.         _connectionString = ConfigurationManager.ConnectionStrings[“BlobStorageConnectionString”].ConnectionString;
  8.         _queueName = ConfigurationManager.AppSettings[“ImagesQueue”];;
  9.     }
  10.     public Task AddMessageToQueueAsync(string messageId, T messageObject)
  11.     {
  12.         // empty for now
  13.     }
  14. }

 

Wiring the Controller

Now, let’s jump into our HomeController and call the IQueueService’s method to add the message to the queue.  We’ll first need a concrete implementation of IQueueService. Here again for sake of demo, I am just creating a QueueService directly and skipping Dependency Injection.

HomeController.cs
  1. public class HomeController : Controller
  2. {
  3.     private readonly IImageService _imageService = new ImageService();
  4.     private readonly IQueueService<UploadedImage> _queueService = new QueueService<UploadedImage>();
  5.     public ActionResult Index()
  6.     {
  7.         return View(new UploadedImage());
  8.     }
  9.     [HttpPost]
  10.     public async Task<ActionResult> Upload(FormCollection formCollection)
  11.     {
  12.         var model = new UploadedImage();
  13.         if (Request != null)
  14.         {
  15.             HttpPostedFileBase file = Request.Files[“uploadedFile”];
  16.             model = await _imageService.CreateUploadedImage(file);
  17.             await _imageService.AddImageToBlobStorageAsync(model);
  18.             await _queueService.AddMessageToQueueAsync(model.Name, model);
  19.         }
  20.         return View(“Index”, model);
  21.     }
  22. }

 

Serializing the Message for the Queue

One important thing to note is that messages that are posted to Azure Queue service have a maximum size of 65k.  If you recall from the first Azure Bit, I mentioned that we would use the UploadedImage class as the payload of our Azure Queue message.  However, if we just push the UploadedImage directly to the Azure queue, we will get an Exception since the Data property will almost always exceed 65k as it contains a full byte array copy of the image.  We do not even need that copy of the image in our Queue message.  The UploadedImage class contains the Url where the image is stored in Azure and that’s all we need to get our original image for processing.  There are numerous ways we could deal with this Data property, such as using a separate object to upload to the queue that didn’t have this property, removing/replacing the byte array before saving, etc.  What I have chosen to do instead is simply indicate that the Data property should not be serialized by replacing the Auto property we had for Data with an explicit property and backing field and then adding the [NonSerialized] attribute to the backing field to indicate that we do not want this data to be included in the serialization process.  You’ll also need to add the [Serialized] attribute to the UploadedImage and Thumbnail classes as we will ultimately be posting the UploadedImage to the queue service in the form of a byte array which we will serialize shortly.

UploadedFile.cs
  1. [Serializable]
  2. public class UploadedImage
  3. {
  4.     public UploadedImage()
  5.     {
  6.         // hard-coded to a single thumbnail at 200 x 300 for now
  7.         Thumbnails = new List<Thumbnail> { new Thumbnail { Width = 200, Height = 300 } };
  8.     }
  9.     public string Name { get; set; }
  10.     public string ContentType { get; set; }
  11.     // no need to serialize the file bytes of the image to the queue (will be larger than 65k max)
  12.     [NonSerialized]
  13.     private byte[] _data;
  14.     public byte[] Data
  15.     {
  16.         get { return _data; }
  17.         set { _data = value; }
  18.     }
  19.     public string Url { get; set; }
  20.     public List<Thumbnail> Thumbnails { get; set; }
  21. }

 

Thumbnail.cs
  1. [Serializable]
  2. public class Thumbnail
  3. {
  4.     public int Width { get; set; }
  5.     public int Height { get; set; }
  6.     public string Url { get; set; }
  7. }

 

Adding the Message to the Queue

Now that we have the outer plumbing in place to insert the UploadedImage into the Azure queue, let’s revisit the AddMessageToQueueAsync method on QueueService where the actual interaction with the Azure queue takes place.  First, we get the queue via a private GetQueue method (more on that below).  Then, we serialize the object that will be the payload of our Azure message (in our case, the UploadedImage).  Finally, we create the actual message for the queue and call AddMessageAsync to insert it into our Azure queue.  The private GetQueue method of QueueService bears some explanation.  To get access to a queue or to create one, you first have to access your storage account and use that to create a CloudQueueClient and then finally, you can use the queueClient to get a reference to a specific queue by name.  Additionally, you can request that the queue be created if it does not already exist by calling the queue.CreateIfNotExists method as I have done here.

 

QueueService.cs
  1. public Task AddMessageToQueueAsync(string messageId, T messageObject)
  2. {
  3.     var queue = GetQueue();
  4.     // serialize the payload for the message
  5.     var serializedMessage = messageObject.SerializeToByteArray();
  6.     // Create the actual queue message
  7.     CloudQueueMessage message = new CloudQueueMessage(serializedMessage);
  8.     // Add the message to the queue
  9.     return queue.AddMessageAsync(message);
  10. }
  11. private CloudQueue GetQueue()
  12. {
  13.     // get the storage account
  14.     CloudStorageAccount storageAccount = CloudStorageAccount.Parse(_connectionString);
  15.     // create the queue client
  16.     CloudQueueClient queueClient = storageAccount.CreateCloudQueueClient();
  17.     // get a reference to the queue
  18.     CloudQueue queue = queueClient.GetQueueReference(_queueName);
  19.     // create the queue if it does not exist
  20.     queue.CreateIfNotExists();
  21.     return queue;
  22. }

 

Extension Method for ByteArray

If you tried to compile now, you’d find that the SerializeToByteArray method does not exist.  This is an extension method that I added, along with a complementary Deserialize method, to a static ByteArrayExtensions class.  We’ll use the Deserialize method in Azure Bit #4 when we read the message from the queue and need to re-hydrate it as our UploadedImage.

ByteArrayExtensions.cs
  1. public static class ByteArrayExtensions
  2. {
  3.     public static byte[] SerializeToByteArray(this object obj)
  4.     {
  5.         if (obj == null)
  6.         {
  7.             return null;
  8.         }
  9.         var bf = new BinaryFormatter();
  10.         using (var ms = new MemoryStream())
  11.         {
  12.             bf.Serialize(ms, obj);
  13.             return ms.ToArray();
  14.         }
  15.     }
  16.     public static T Deserialize<T>(this byte[] byteArray) where T : class
  17.     {
  18.         if (byteArray == null)
  19.         {
  20.             return null;
  21.         }
  22.         using (var memStream = new MemoryStream())
  23.         {
  24.             var binForm = new BinaryFormatter();
  25.             memStream.Write(byteArray, 0, byteArray.Length);
  26.             memStream.Seek(0, SeekOrigin.Begin);
  27.             var obj = (T)binForm.Deserialize(memStream);
  28.             return obj;
  29.         }
  30.     }
  31. }

 

The Final Result – Message in the Queue

At this point, you should be able to run the application and select an image which gets uploaded to Azure Blob Storage and you should have a message sitting in your Azure queue telling you that the image is uploaded and waiting for processing.  There are a couple of ways to view what’s in your queue without having to actually fetch it.  As of this writing, Azure Storage Explorer seems to have issues with byte array messages and just gives errors when I try to view them there.  Probably the most convenient place to get a quick look into whether my message made it or not is via the Server Explorer directly in Visual Studio.

image

 

You can see here that my message was inserted into the queue.  Additionally, you can view the contents of your Blobs in Server Explorer to verify that the image itself is in Azure Blob Storage. Go ahead and publish your latest to Azure by right-clicking on the web project in Solution Explorer and choosing “Publish..”.

In Azure Bit #4 – Adding the Azure Web Job, we will create an Azure Web Job that pulls our message from the queue and gets it ready for processing.

Did you miss the first Azure Bit?  You can start the series for Azure Bits here: Azure Bits #1 – Up and Running at the Wintellect DevCenter.

 

Badge-Azure

Stay Informed

Sign up for the latest blogs, events, and insights.

We deliver solutions that accelerate the value of Azure.
Ready to experience the full power of Microsoft Azure?

Atmosera is thrilled to announce that we have been named GitHub AI Partner of the Year.

X