How to create a BitmapImage from a pixel byte array (live video display)

Question

I need to display live images on a WPF control. I'm looking for the fastest way to do this using WPF.

I'm capturing images from a camera using its dll API (AVT).

The image is writen by the dll and the camera rises a callback with an IntPtr to a Image struct called tFrame (described below). The pixel data is stored at the ImageBuffer propertie with is an InPtr to a byte array.

I know how to create a Bitmap from the pixel byte array, but not a BitmapImage. So it is possible to create a Bitmap and then create a BitmapImagem from it. Here there is a way to create a BitmapImage from a Bitmap on the memory. But I want to create the BitmapImage directly from the data source (tFrame). How can I do that?

I know that BitmapImage have a CopyPixels method, but it laks of a SetPixels.

public struct tFrame
{
    public IntPtr AncillaryBuffer;
    public uint AncillaryBufferSize;
    public uint AncillarySize;
    public tBayerPattern BayerPattern;
    public uint BitDepth;
    public tFrameCtx Context;
    public tImageFormat Format;
    public uint FrameCount;
    public uint Height;
    public IntPtr ImageBuffer;
    public uint ImageBufferSize;
    public uint ImageSize;
    public uint RegionX;
    public uint RegionY;
    public tErr Status;
    public uint TimestampHi;
    public uint TimestampLo;
    public uint Width;
}

Here is how I create a Bitmap from a pixel byte array. This was used at the WinForm version of the software.

private void CreateBitmap(tFrame frame)
{
    //This sample is for a 8bpp captured image
    PixelFormat pxFormat = PixelFormat.Format8bppIndexed;

    //STRIDE
    //[https://stackoverflow.com/questions/1983781/why-does-bitmapsource-create-throw-an-argumentexception/1983886#1983886][3]
    //float bitsPerPixel = System.Drawing.Image.GetPixelFormatSize(format);
    int bitsPerPixel = ((int)pxFormat >> 8) & 0xFF;
    //Number of bits used to store the image data per line (only the valid data)
    int validBitsPerLine = ((int)frame.Width) * bitsPerPixel;
    //4 bytes for every int32 (32 bits)
    int stride = ((validBitsPerLine + 31) / 32) * 4;

    Bitmap bmp = new Bitmap((int)frame.Width, (int)frame.Height, stride, pxFormat, frame.ImageBuffer);
}

EDIT 1:

Thanks to dr.mo, now I'm able to display 60 FPS 1024x1024 images with ~3% CPU usage! What I'm doing is:

//@ UI Thread
public WriteableBitmap wbm = new WriteableBitmap(1024, 1024, (double)96, (double)96, System.Windows.Media.PixelFormats.Gray8, null);
this.wbBackBuffer = this.wbm.BackBuffer;

//This can be called by a timer in the UI thread or at the grab Thread for every image, the CPU usage is almost the same.
void UpdateDisplayImage()
{
wbm.Lock();
wbm.AddDirtyRect(new Int32Rect(0, 0, wbm.PixelWidth, wbm.PixelHeight));
wbm.Unlock();
}

//@ Grab Thread
//Update the backbuffer with new camera image data.
UpdateBackBuffer(...);

/// <summary>
/// [http://msdn.microsoft.com/en-us/library/system.windows.media.imaging.writeablebitmap.aspx]
/// </summary>
public void UpdateBackBuffer(IntPtr pData, int w, int h, int ch)
{
    //Can not acess wbm from outside UI thread
    //CopyMemory(wbm.BackBuffer, pData, (uint)(w * h * ch));
    //I dont know if it is safe to write to it buffer like this:
    CopyMemory(this.wbBackBuffer, pData, (uint)(w * h * ch));
}

msdn.microsoft.com/en-gb/library/system.windows.interop.imaging.createbitmapsourcefrommemorysection.aspx — DarkSquirrel42, Apr 25 '13 at 16:55

morishuz · Accepted Answer · 2013-04-25T17:21:12.783

15

This should do the trick. it's super fast.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Windows;
using System.Windows.Controls;
using System.Windows.Data;
using System.Windows.Documents;
using System.Windows.Input;
using System.Windows.Media;
using System.Windows.Media.Imaging;
using System.Windows.Navigation;
using System.Windows.Shapes;

using System.Drawing;
using System.Runtime.InteropServices;
using System.IO;
using System.ComponentModel;


public class MakeBitmapSource
{
    [DllImport("kernel32.dll", EntryPoint = "RtlMoveMemory")]
    public static extern void CopyMemory(IntPtr Destination, IntPtr Source, uint Length);



    public static BitmapSource FromNativePointer(IntPtr pData, int w, int h, int ch)
    {
        PixelFormat format = PixelFormats.Default;

        if (ch == 1) format = PixelFormats.Gray8; //grey scale image 0-255
        if (ch == 3) format = PixelFormats.Bgr24; //RGB
        if (ch == 4) format = PixelFormats.Bgr32; //RGB + alpha


        WriteableBitmap wbm = new WriteableBitmap(w, h, 96, 96, format, null);
        CopyMemory(wbm.BackBuffer, pData, (uint)(w * h * ch));

        wbm.Lock();
        wbm.AddDirtyRect(new Int32Rect(0, 0, wbm.PixelWidth, wbm.PixelHeight));
        wbm.Unlock();

        return wbm;
    }

    public static BitmapSource FromArray(byte[] data, int w, int h, int ch)
    {
        PixelFormat format = PixelFormats.Default;

        if (ch == 1) format = PixelFormats.Gray8; //grey scale image 0-255
        if (ch == 3) format = PixelFormats.Bgr24; //RGB
        if (ch == 4) format = PixelFormats.Bgr32; //RGB + alpha


        WriteableBitmap wbm = new WriteableBitmap(w, h, 96, 96, format, null);
        wbm.WritePixels(new Int32Rect(0, 0, w, h), data, ch * w, 0);

        return wbm;
    }
}

edited Apr 25 '13 at 17:21

answered Apr 25 '13 at 17:05

morishuz

2,302
4
21
21

and call it like this: xamlImage.Source = MakeBitmapSource.FromNativePointer(tFrame.ImageBuffer,tFrame.Width,tFrame.Height,tFrame.BitDepth/8); – morishuz Apr 25 '13 at 17:13
Very nice. I'm going to test it now! – Pedro77 Apr 25 '13 at 17:20
you can update the backbuffer from a non-UI thread too! just make sure to freeze wbm and to call wbm.AddDirtyRect from the UI thread once you are done. i use this to display webcam video. works great. – morishuz Apr 25 '13 at 17:22
Very nice! Have you tried this solution: http://stackoverflow.com/questions/15274699/create-a-bitmapimage-from-a-byte-array/15290190#15290190 ? It returns a BitmapSource too. I dont know if it is faster than your solution. – Pedro77 Apr 25 '13 at 17:37
1

no i haven't but i am aware of it and i bet it is slower because it involves image creation at every step. using WB you only need to copy the data not create a new image everytime you update your video frame. plus the issue is 1. you don't have a byte array but a native pointer (IntPtr) 2. you can update in a non UI thread using WB - kind of essential for what you are trying to do. this is the absolute fastest way to display a bitmap in WPF that i've come across. – morishuz Apr 25 '13 at 17:51
can you update WriteableBitmap once it has been assigned to Source? On a seperate thread? Or do you just create new BitmapSource each frame? – Erti-Chris Eelmaa Apr 25 '13 at 19:26
Using this static method I loss acess to the WriteableBitmap, so I can not dispose it and the app runs auto of memory realy fast. I think the solution woukd be: 1. Keep a reference to the WriteableBitmap or 2. as dr.mo said, use the same WriteableBitmap for all the images. @ solution 2. I see a problem when you want to read pixels (eg: at mouse position) becouse you can get read/write acess violation. To solve that, I think you can make a copy of the image pixels data or use lock around the image change (write) and getting image pixels (read). – Pedro77 Apr 26 '13 at 12:05
1

@Erti-Chris Eelmaa yes you can update WB once it has been assigned to source! what you do is you freeze WB, get a reference to it's native BackBuffer property and update this anywhere you want to, including a non UI thread. however, the image wont update unless you call the AddDirtyRect function in the UI thread. that's it. – morishuz Apr 27 '13 at 08:54
dr.mo, please take a look below the EDIT at my question. I was not able to "Freeze" the WB, if I do it, I'll get access denied exception when trying to update its BackBuffer outside UI. To update the BackBuffer I need to first save the IntPtr at the UI and use it to copy memory to it, because if I try to access the wb.BackBuffer from a thread different than UI, I'll get a cross thread exception. Can you please clarify how exactly you are doing it? thanks! – Pedro77 May 28 '13 at 17:11
make a local copy of the back buffer reference. i.e. IntPtr buffer = wb.BackBuffer - then just use that instead. – morishuz Jul 15 '13 at 12:17
Yeah I just did that, but if you Freeze the WB, you can NOT updated it using AddDirtRect, because an exceptions is razed. My WB is not freezed, I update the buffer IntPtr copy and update the screen using Lock, AddDirtRect, Unlock. – Pedro77 Jul 23 '13 at 16:13

How to create a BitmapImage from a pixel byte array (live video display)

1 Answers1

Linked