2

Question:

What is a fast way to scale and/or crop a bitmap provided from a WritableBitmap for display in the UI?

Requirements:

  1. Have Low CPU usage
  2. Handle large images (5 Megapixel, abt 2500x2000 pixels)
  3. Resize and/or crop to the same resolution/area as the UI element the bitmap is displayed in.
  4. Use WPF

Specifically, it must allow a 14FPS 5 Megapixel camera image stream to be displayed in a WPF UI element at full speed.

Update:

I have been able to speed up the drawing quite a bit by painting to a Canvas control using an ImageBrush as follows, where m_bitmap is my WriteableBitmap:

ImageBrush brush = new ImageBrush();
brush.ImageSource = m_bitmap;
brush.Stretch = Stretch.Uniform;
canvas.Background = brush;

I'm now able to get the full 14FPS, though it still using about 20% CPU, so I'm not sure how well it preform if I add another camera or two (the plan is to have 4 or so running).

Another thing I think might be slowing down the drawing is the images are in a mono, Gray8, format, not the standard RGB32 (or is it bgra32 for WPF?) format. If I understand correctly, the image has to be converted to the standard format to be displayed, which would add significant overhead to each frame's drawing time.

Some background:

I'm currently working with a 5 Megapixel, 14 FPS, video camera and am trying to get the frames to render to the screen at full speed. I would like to do this using WPF.

I currently have an example in WinForms that runs full speed for an unscaled image, but (as I would expect) it has major trouble if I set the pictureBox.SizeMode = Zoom;. The example reads raw data directly from the camera stream to a buffer and then copies the data from the buffer into the bitmap set to the PictureBox control. The copy algorithm uses LockBits to speed things up.

I converted that example into WPF, rewriting the parts using Bitmap objects to instead use WritableBitmap objects and an Image control instead of PictureBox. Unforunately this is not able to render the stream to the screen at any decent rate, scaled or unscaled. Both have significant CPU load and very slow updates.

The performance when rendering to the screen is turned off is great. It is able to process the image stream at full speed and resolution while using around 3% CPU and less than 100MB memory.

Note: when I say rendering to the screen is turned off, the WritableBitmap is still being continuously updated, only is not set to the Image control.

I've seen a lot of discussion about getting fast bitmap updating in WPF, but have been unsuccessful in getting it to work at an reasonable speed/cpu load. Also I would like to have the image scaled in such a way that I can see the whole image.

I imagine the key will lie in some sort of scaling/crop combination that needs to be done so that WPF will not try to render(cache?) all 5 million pixels, but only those on the screen, and only at the current screen resolution. I imagine/hope this can be done fairly easily and without too much hit to memory or CPU, but currently have no idea how to do so. I have found the DecodePixelWidth and DecodePixelHeight properties, but those are only applicable when loading an image from a file to a BitmapImage.

Community
  • 1
  • 1
Scott Lemmon
  • 694
  • 7
  • 16
  • You might provide some code that shows how you decode the pixel buffer and update the bitmap. Maybe there's something to improve. – Clemens Jan 23 '13 at 08:50
  • Those are both quite fast. As I explained above, when I turn off the rendering to screen, the CPU usage drops to 3%... there might be a little I can do to improve it, but that is not where the major bottleneck is right now. The major problem is in the drawing to screen. – Scott Lemmon Jan 23 '13 at 23:56

2 Answers2

0

Did you have a look at the following post?

Resizing WritableBitmap

If it does not solve your problem, I have more questions for you:

What is the resolution of your image? Is the size of you UI element constant? What's its size?

Edit: After your edit, I noticed that you want to display a BitmapImage in Gray8 PixelFormat, why don't you try to set this property when creating your BitmapImage (m_bitmap)? m_bitmap.Format = PixelFormat.Gray8; // could not test

I am certain that taking your 8 bits/pixel and multiplying the amount of bits needed per pixel by 4 while not gaining any quality is slowing down your application. Especially because you run operations on 32 bits per pixel images when you could be running those operations on 8 bits per pixel images.

Community
  • 1
  • 1
Pierre-Luc
  • 720
  • 5
  • 13
  • Thank you for the information... Unfortunately the linked information didn't really help me, but it did get me going in a direction that did. Check out the update for more details. – Scott Lemmon Jan 23 '13 at 08:00
  • @ScottLemmon I've updated my answer, hopefully it helps. Basically, if you are displaying grayscale images, no need to make your image 4 times bigger, this will slow down operations. – Pierre-Luc Jan 23 '13 at 15:57
  • It's not that I want to display in Gray8, it's that that is what I'm currently using. The raw data from the camera is an 8bit mono stream. I'm decoding that data into a `WriteableBitmap` with the `PixelFormats.Gray8` Format property. What I was saying in the update is that I remember hearing that WPF draws everything in a 32bit color format, so any bitmap that isn't in that format must be converted to it before drawing. If that really is the case it would probably slow down the drawing quite a bit. – Scott Lemmon Jan 24 '13 at 00:05
0

While its interface is a bit old-fashioned, I believe that convert (see http://en.wikipedia.org/wiki/ImageMagick) is very often used (and may in fact be the industry standard).

Edit: StackOverflow has about 2,300 question tagged with imagemagick here. See for example What is the difference for sample/resample/scale/resize/adaptive-resize/thumbnail operators in ImageMagick convert?

The OP for https://apple.stackexchange.com/a/41531 decided to go with ImageMagick. And the accepted answer to Efficient JPEG Image Resizing in PHP also suggests ImageMagick, with 19 votes.

However, I don't know whether ImageMagick is capable of meeting your requirements of 14FPS, 5 Megapixels.

The only answer to Recommendation for real time image processing tools on Linux suggests a fork graphicsmagick, which seems to also be available for Windows.

Community
  • 1
  • 1
Joseph Quinsey
  • 9,553
  • 10
  • 54
  • 77
  • I don't understand what you are suggesting... that I use the linked software to convert the image stream? I checked it out, and while it does have a .net api, it has almost no documentation. Additionally, I don't need something to convert my bitmap to a different format, I need something to resize it very efficiently. Please clarify. – Scott Lemmon Jan 23 '13 at 08:09
  • Thank you for the extra information. From what I've gathered, ImageMagick only converts from a file, or in .net's case I imagine maybe an IO stream would work as well. Unfortunately this does not meet my requirements. As I stated in the first line of my post, I need to resize directly from a WriteableBitmap (or some other sort of BitmapSource is ok too). – Scott Lemmon Jan 25 '13 at 02:45