How can I show Kinect video stream in Windows.Forms.PictureBox appropriately?

Question

I'm trying to display Kinect's video stream into PictureBox. The reason is, I'd like to overlay it with some images and use FillEllipse() method to add real time markers. However, I ended up with a box with a red x (cross) in it. Could someone show me, where did I go wrong? Should I use WritableBitmap instead? I had a thought of this, but Writeable bitmap does not offer method such as FillEllipse() to place markers.

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Linq;
using System.Text;
using System.Windows.Forms;

using Microsoft.Kinect;
using System.Drawing.Imaging;
using System.Drawing;
using System.Runtime.InteropServices;

namespace fTrack_WF
{
    public partial class Form1 : Form
    {
        public Form1()
        {
            InitializeComponent();
        }

        KinectSensor myKinect;

        private void Window_Loaded(object sender, EventArgs e)
        {
            if (KinectSensor.KinectSensors.Count == 0)
            {
                MessageBox.Show("No Kinects device detected", "Camera View");
                Application.Exit();
                return;
            }

            try
            {
                // get first Kinect device attached on computer
                myKinect = KinectSensor.KinectSensors[0];

                // enable depth stream
                myKinect.DepthStream.Enable();

                // enable color video stream
                myKinect.ColorStream.Enable();

                // start the sensor
                myKinect.Start();


                // connect up the video event handler
                myKinect.ColorFrameReady += new EventHandler<ColorImageFrameReadyEventArgs>(myKinect_ColorFrameReady);

            }
            catch
            {
                MessageBox.Show("Kinect initialise failed", "Camera viewer");
                Application.Exit();
            }


        }


        #region Video Image Processing

        byte[] colorData = null;
        Bitmap kinectVideoBitmap = null;
        IntPtr colorPtr;

        void myKinect_ColorFrameReady(object sender, ColorImageFrameReadyEventArgs e)
        {
            using (ColorImageFrame colorFrame = e.OpenColorImageFrame())
            {
                if (colorFrame == null) return;

                if (colorData == null)
                    colorData = new byte[colorFrame.PixelDataLength];

                colorFrame.CopyPixelDataTo(colorData);

                Marshal.FreeHGlobal(colorPtr);
                colorPtr = Marshal.AllocHGlobal(colorData.Length);
                Marshal.Copy(colorData, 0, colorPtr, colorData.Length);

                kinectVideoBitmap = new Bitmap(
                    colorFrame.Width,
                    colorFrame.Height,
                    colorFrame.Width * colorFrame.BytesPerPixel;
                    PixelFormat.Format32bppRgb,
                    colorPtr);

                kinectVideoBox.Image = kinectVideoBitmap;

                kinectVideoBitmap.Dispose();

            }

        }

        #endregion
    }
}

Thank you very much!

Regards, ikel

score 2 · Answer 1 · answered Sep 26 '12 at 21:38

I'm not clear why you are using a WinForms PictureBox over just using WPF.

Have you tried placing a Canvas on top of the video stream, demonstrated in the SDK examples, and simply drawn to that?

    <Grid HorizontalAlignment="Right" VerticalAlignment="Bottom" Width="320" Height="240">
        <lt:KinectDepthViewer x:Name="DepthViewer" KinectSensorManager="{Binding KinectSensorManager}" />
        <Canvas>
            <lt:KinectSkeletonViewer
                                KinectSensorManager="{Binding KinectSensorManager}"
                                Width="{Binding ElementName=DepthViewer, Path=ActualWidth}"
                                Height="{Binding ElementName=DepthViewer, Path=ActualHeight}"
                                ShowBones="True" ShowJoints="True" ShowCenter="True" ImageType="Depth" />
        </Canvas>
    </Grid>


    <Canvas Name="DrawingCanvas">
    </Canvas>

The second canvas is at a higher z-index and any object on that will cover up your video stream.

P.S. Although my code points to the depth viewer, the video stream is done in the same way when using the examples from the SDK.

@ECM, to be honest, the deadline of the project is rather tight, overall budget is rather short, and my knowledge of WPF and XAML is rather limited. I can do things slightly quicker with WinForms. — ikel, Sep 27 '12 at 22:23
Ah. My knowledge in the WinForms realm is limited at best. Hope you are able to find the issue! — Nicholas Pappas, Sep 28 '12 at 16:26

score 2 · Accepted Answer · edited May 23 '17 at 11:54

I found the answer. Dispose is necessary to free up resources as indicated here. The problem was, I disposed too early after drawing it, and so it appeared as if nothing was drawn. However, the clearer answer, to me anyway, was given here.

Bitmap inherits from Image, which implements IDisposable, so when you're done using an instance, you should call Dispose() on it. This will clean up the unmanaged resource in Image.

However, Image also implements a finalizer, so if for some reason you cannot call Dispose(), the resource will be reclaimed during finalization of the instance, which will happen at some point after the instance is no longer referenced.

I simply removed kinectVideoBitmap.Dispose(); and my Windows Forms displays Kinect's video stream inside PictureBox control.

Regards, ikel

How can I show Kinect video stream in Windows.Forms.PictureBox appropriately?

2 Answers2