Silverlight’s Big Image Problem (and What You Can Do About It)

Quick: Can you spot the problem with these three lines of code?

BitmapImage bi = new BitmapImage();

bi.SetSource(stream);

TheImage.Source = bi;

These statements create an image from a stream of PNG or JPG image bits and display the image by assigning it to a XAML Image object named TheImage. It’s boilerplate code used to display images read from the local file system or obtained from a service. And while there’s nothing inherently wrong with the code itself, you’ll want to think carefully before including it in any Silverlight application.

I call it “Silverlight’s Big Image Problem.” Not the kind of image problem a movie star might suffer, but an inherent memory-consumption problem when dealing with large bitmap images in Silverlight.

The problem manifests itself when you handle large images in large numbers. The My Pictures Viewer that I blogged about yesterday is a case in point. When a user running the application selects a folder containing one or more image files, the viewer displays clickable thumbnail versions of the images. The problem is that because Silverlight’s BitmapImage class consumes massive amounts of memory (up to 40 or 50 MB per image for a typical 2 to 3 MB digital photo), you simply can’t have too many instances extant at once. But to create a thumbnail, you first need a BitmapImage that wraps the entire image. You might create a thumbnail by assigning the BitmapImage to an Image object that measures just 100 by 100 pixels, but if the original image measures 4,000 by 4,000 pixels, it’s the latter figure you pay the price for.

To demonstrate, I wrote a simple test harness that you can easily duplicate yourself. I began with an app that pops up an OpenFileDialog and lets the user select an image file from his or her hard disk. Once the image file is selected, the application generates a thumbnail version of the image and adds it to the scene. Then it generates another thumbnail, and then another, and so on and so forth until Silverlight throws an out-of-memory exception. Here is the helper method that I initially used to generate the thumbnails:

private Image CreateThumbnailImage(Stream stream, int width)

{

    BitmapImage bi = new BitmapImage();

    bi.SetSource(stream);

    double cx = width;

    double cy = bi.PixelHeight * (cx / bi.PixelWidth);

    Image image = new Image();

    image.Width = cx;

    image.Height = cy;

    image.Source = bi;

    return image;

}

And here’s what happened when I ran the application and selected a 3,648 x 2,736 JPG with a file size of 2.1 MB:

So get this. The application created 26 thumbnails, each measuring a mere 100 x 75 pixels. But attempting to create a 27th thumbnail produced an out-of-memory exception. When the exception occurred, Task Manager showed that the process’s working set size had grown from 30 MB to nearly 1.5 GB! It seems crazy on the surface, because a full-color 100 x 75 image should only require about 30K of memory. But it makes a lot more sense when you realize that underlying each thumbnail is a gigantic BitmapImage that retains the full fidelity of the 3,648 x 2,736 original.

The obvious question is what do you do about it? Is there a way to efficiently create thumbnail images from streams of image bits in Silverlight? It’s a question that pops up time and again in discussion forums and on message boards. And the short answer is yes, there is a way. But the answer probably isn’t the one you expect.

Developers commonly attempt a solution along these lines:

private Image CreateThumbnailImage(Stream stream, int width)

{

    BitmapImage bi = new BitmapImage();

    bi.SetSource(stream);

    double cx = width;

    double cy = bi.PixelHeight * (cx / bi.PixelWidth);

    Image image = new Image();

    image.Source = bi;

    WriteableBitmap wb = new WriteableBitmap((int)cx, (int)cy);

    ScaleTransform transform = new ScaleTransform();

    transform.ScaleX = cx / bi.PixelWidth;

    transform.ScaleY = cy / bi.PixelHeight;

    wb.Render(image, transform);

    wb.Invalidate();

    Image thumbnail = new Image();

    thumbnail.Width = cx;

    thumbnail.Height = cy;

    thumbnail.Source = wb;

    return thumbnail;

}

The basic idea is that instead of creating a thumbnail by assigning a large BitmapImage to a small Image, you use WriteableBitmap.Render with a ScaleTransform to create a thumbnail, and then assign the WriteableBitmap to an Image. Meanwhile, the BitmapImage and the Image it was temporarily assigned to—the one passed to WriteableBitmap.Render—go out of scope and are eventually picked up by the garbage collector.

It works well in theory, but not so well in practice, thanks to an undocumented behavior of WriteableBitmap. In fact, when I plugged the revised CreateThumbnailImage method into my test harness, the application ran out of memory just as quickly as before.

The problem, it turns out, is that when you call WriteableBitmap.Render, WriteableBitmap apparently retains a reference to the XAML object passed in the first parameter. (I was stumped, too, until Jeffrey Richter and I did a little detective work and discovered what was happening under the hood. Jeffrey’s my go-to guy for CLR issues, and I’m not sure I would have ever figured this out without him asking the right questions and suggesting solutions.) When CreateThumbnailImage returns an Image holding a reference to a WriteableBitmap, and the WriteableBitmap holds a reference to an Image, and the Image holds a reference to a BitmapImage, none of these objects gets garbage-collected. It seems that WriteableBitmap does nothing to solve the problem, especially given that there’s no public method or property you can use to force the WriteableBitmap to release the reference.

But all is not lost. You can make a copy of the WriteableBitmap and assign it to the Image you return. And since you didn’t call Render on the copy, it doesn’t hold a reference to an Image that prevents the garbage collector from cleaning up the BitmapImage. Here is the fixed and final version of CreateThumbnailImage—this time, one that accomplishes what we set out to do:

private Image CreateThumbnailImage(Stream stream, int width)

{

    BitmapImage bi = new BitmapImage();

    bi.SetSource(stream);

    double cx = width;

    double cy = bi.PixelHeight * (cx / bi.PixelWidth);

    Image image = new Image();

    image.Source = bi;

    WriteableBitmap wb1 = new WriteableBitmap((int)cx, (int)cy);

    ScaleTransform transform = new ScaleTransform();

    transform.ScaleX = cx / bi.PixelWidth;

    transform.ScaleY = cy / bi.PixelHeight;

    wb1.Render(image, transform);

    wb1.Invalidate();

    WriteableBitmap wb2 = new WriteableBitmap((int)cx, (int)cy);

    for (int i = 0; i < wb2.Pixels.Length; i++)

        wb2.Pixels[i] = wb1.Pixels[i];

    wb2.Invalidate();

    Image thumbnail = new Image();

    thumbnail.Width = cx;

    thumbnail.Height = cy;

    thumbnail.Source = wb2;

    return thumbnail;

}

When I plugged this implementation into my test harness, it successfully created hundreds of thumbnails (and could have created hundreds, perhaps thousands, more) without significantly increasing the working set size—and without throwing out-of-memory exceptions.

The moral is that you should be very careful about how you use BitmapImage in Silverlight. Even one of them can swell the working set size dramatically, but a couple dozen of them wrapping digital photographs is more than most PCs can handle. With a little care, however, you can scale down the impact of BitmapImage so that memory consumption is proportional to the sizes of the images you’re displaying rather than the sizes of the original, unreduced images. And that, in the end, is a handy arrow to have in your arsenal.