C# WPF AForge : How to make video smooth?

Question

I'm a C# programmer(WPF) and I have a live video image process problem.

MY ENGLISH IS VERY BAD, I will try my best to descript my question:

I use following code to play video with AForge.net :

// persume resolution is 640x480
// presume snapshot is not available
_cameraDevice.OnNewVideoFrame += NewVideoFrame;

In NewVideoFrame, I give the Bitmap object to a observable Bitmap model:

private Bitmap _liveView = null;
public Bitmap LiveView { get { return _liveView; } set { SetProperty(ref _liveView, value); } }

private void NewVideoFrame(object sender, AForge.Video.NewFrameEventArgs eventArgs)
{
    LiveView = (Bitmap)eventArgs.Frame.Clone();
}

Then I use a Bitmap to ImageSource Converter to convert bitmap to WPF Image control:

// Bitmap to ImageSource Converter
public class BitmapToImageSourceConverter : IValueConverter
{
    public object Convert(object value, Type targetType, object parameter, System.Globalization.CultureInfo culture)
    {
        if (value == null)
            return null;

        try
        {
            System.Drawing.Bitmap bmp = (System.Drawing.Bitmap)value;

            System.IO.MemoryStream ms = new System.IO.MemoryStream();
            if (bmp.RawFormat.Guid == System.Drawing.Imaging.ImageFormat.MemoryBmp.Guid)
                bmp.Save(ms, System.Drawing.Imaging.ImageFormat.Bmp);
            else
                bmp.Save(ms, bmp.RawFormat);
            ms.Seek(0, System.IO.SeekOrigin.Begin);
            System.Windows.Media.Imaging.BitmapImage bi = new System.Windows.Media.Imaging.BitmapImage();
            bi.BeginInit();
            bi.StreamSource = ms;
            bi.EndInit();
            return bi;
        }
        catch
        {
            return null;
        }

    }

    public object ConvertBack(object value, Type targetType, object parameter, System.Globalization.CultureInfo culture)
    {
        throw new NotImplementedException();
    }

}

At last, my xaml like this:

<Image Source="{Binding LiveView, Converter={StaticResource BitmapToImageSourceConverter}}"/>

Then I run the application, everything looks good. BUT, when I try to add some logic to NewVideoFrame, the video become very slow:

private void NewVideoFrame(object sender, AForge.Video.NewFrameEventArgs eventArgs)
{
    LiveView = (Bitmap)eventArgs.Frame.Clone();

    SomeLiveVideoFaceDetectFunction((Bitmap)eventArgs.Frame.Clone()); 
    OrSomeShapeDetectionFunction((Bitmap)eventArgs.Frame.Clone());
    // Such functions like these 2 make video very slow.
    // how slow? maybe 5-10 frames per second with 640x480 reslution
    // Logitec C920 camera, it's a very good device.
}

SomeLiveVideoFaceDetectFunction() is provided by Innovatrics IFace, the live video with face detection is very smooth in their demo they show me, so I believe their SDK is not the problem.

If their SDK's good, then my codes are the problem.

My question are:

Normally how everybody make a live-view-image-processing logic? Is my idea upon right or not?
The Converter converts bitmap to imagesource, is it a bad way to show live image in WPF?

In a nutshell, I want to know the whole thing about how to make a smooth video with image-processing, Thanks.

You should definitely not encode a bitmap into a MemoryStream, then create a new BitmapImage from that stream *on every frame*. Better create a WriteableBitmap *once* and copy raw pixel data into it as shown in [this answer](http://stackoverflow.com/a/30729641/1136211). — Clemens, Jan 05 '17 at 13:29

score 1 · Answer 1 · answered Jan 06 '17 at 05:48

I figure it out myself, use another thread to do image-processing job:

private Bitmap _imageToProcess = null;
private bool _processingImage = false;

private void NewVideoFrame(object sender, AForge.Video.NewFrameEventArgs eventArgs)
{
    _imageToProcess = (Bitmap)eventArgs.Frame.Clone();
    LiveView = (Bitmap)eventArgs.Frame.Clone();
}

private void StartProcessImage()
{
    Thread t = new Thread(ProcessImage);
    t.Start();
    _processingImage = true;
}

private void ProcessImage()
{
    while(_processingImage)
    {
        SomeLiveVideoFaceDetectFunction((Bitmap)_imageToProcess.Clone()); 
        OrSomeShapeDetectionFunction((Bitmap)_imageToProcess.Clone());
    }

    _processingImage = false;
}

Now the frame rate is normal.

Hope this helps beginners of Live video image processing.

C# WPF AForge : How to make video smooth?

1 Answers1