1

I am having a problem with EmguCV. I used a demo application, and edited it to my needs. It involves the following function:

public override Image<Gray, byte> DetectSkin(Image<Bgr, byte> Img, IColor min, IColor max)
        {
            Image<Hsv, Byte> currentHsvFrame = Img.Convert<Hsv, Byte>();
            Image<Gray, byte> skin = new Image<Gray, byte>(Img.Width, Img.Height);
            skin = currentHsvFrame.InRange((Hsv)min,(Hsv)max);
            return skin;
        }

In the demo application, the Image comes from a video. The frame is capured from the video like this:

 Image<Bgr, Byte> currentFrame;
grabber = new Emgu.CV.Capture(@".\..\..\..\M2U00253.MPG");            
            grabber.QueryFrame();
currentFrame = grabber.QueryFrame();

In my application, the Image comes from a microsoft kinect stream.

I use the following function:

private void SensorColorFrameReady(object sender, ColorImageFrameReadyEventArgs e)
        {
            using (ColorImageFrame colorFrame = e.OpenColorImageFrame())
            {
                if (colorFrame != null)
                {
                    // Copy the pixel data from the image to a temporary array
                    colorFrame.CopyPixelDataTo(this.colorPixels);

                    // Write the pixel data into our bitmap
                    this.colorBitmap.WritePixels(
                        new Int32Rect(0, 0, this.colorBitmap.PixelWidth, this.colorBitmap.PixelHeight),
                        this.colorPixels,
                        this.colorBitmap.PixelWidth * sizeof(int),
                        0);

                    Bitmap b = BitmapFromWriteableBitmap(this.colorBitmap);
                    currentFrame = new Image<Bgr, byte>(b);
                    currentFrameCopy = currentFrame.Copy();
                    skinDetector = new YCrCbSkinDetector();

                    Image<Gray, Byte> skin = skinDetector.DetectSkin(currentFrame, YCrCb_min, YCrCb_max);


                }
            }
        }
private static System.Drawing.Bitmap BitmapFromWriteableBitmap(WriteableBitmap writeBmp)
        {
            System.Drawing.Bitmap bmp;
            using (System.IO.MemoryStream outStream = new System.IO.MemoryStream())
            {
                BitmapEncoder enc = new BmpBitmapEncoder();
                enc.Frames.Add(BitmapFrame.Create((BitmapSource)writeBmp));
                enc.Save(outStream);
                bmp = new System.Drawing.Bitmap(outStream);
            }
            return bmp;
        }

Now, the demo application works, and mine doesn't. Mine gives the following exception: exception

And, the image here, contains the following: enter image description here

I really don't understand this exception. And, now, when I run the demo, working aplication, the image, contains: enter image description here

Which is, in my eyes, exactly the same. I really don't understand this. Help is very welcome!

Arnout
  • 657
  • 4
  • 12
  • 23

1 Answers1

1

To make things easier I've uploaded a working WPF solution for you to the code reference sourceforge page I've been building:

http://sourceforge.net/projects/emguexample/files/Capture/Kinect_SkinDetector_WPF.zip/download https://sourceforge.net/projects/emguexample/files/Capture/

This was designed and tested using EMGU x64 2.42 so in the Lib folder of the project you will find the referenced dlls. If you are using a different version you will need to delete the current references and replace them with the version you're using.

Secondly the project is design like all projects from the code reference library to be built from the Emgu.CV.Example folder into the ..\EMGU 2.X.X.X\bin.. global bin directory where the opencv compiled libraries are within a folder either x86 or x64.

If you struggle to get the code working I can provide all components but I hate redistributing all the opencv files that you already have so let me know if you want this.

You will need to resize the Mainwindow manually to display both images as I didn't spend to much time playing with layout.

So the code...

In the form initialisation method I check for the kinect sensor and set up the eventhandlers for the frames ready. I have left the original threshold values and skinDetector type although I don't use the EMGU version I just forgot to remove it. You will need to play with the threshold values and so on.

            //// Look through all sensors and start the first connected one.
            //// This requires that a Kinect is connected at the time of app startup.
            //// To make your app robust against plug/unplug, 
            //// it is recommended to use KinectSensorChooser provided in Microsoft.Kinect.Toolkit (See components in Toolkit Browser).
            foreach (var potentialSensor in KinectSensor.KinectSensors)
            {
                if (potentialSensor.Status == KinectStatus.Connected)
                {
                    this.KS = potentialSensor;
                    break;
                }
            }

            //If we have a Kinect Sensor we will set it up
            if (null != KS)
            {
                // Turn on the color stream to receive color frames
                KS.ColorStream.Enable(ColorImageFormat.RgbResolution640x480Fps30);
                //Turn on the depth stream to recieve depth frames
                KS.DepthStream.Enable(DepthImageFormat.Resolution640x480Fps30);

                //Start the Streaming process
                KS.Start();
                //Create a link to a callback to deal with the frames
                KS.AllFramesReady += new EventHandler<AllFramesReadyEventArgs>(KS_AllFramesReady);

                //We set up a thread to process the image/disparty map from the kinect
                //Why? The kinect AllFramesReady has a timeout if it has not finished the streams will simply stop
                KinectBuffer = new Thread(ProcessBuffer);

                hsv_min = new Hsv(0, 45, 0);
                hsv_max = new Hsv(20, 255, 255);
                YCrCb_min = new Ycc(0, 131, 80);
                YCrCb_max = new Ycc(255, 185, 135);

                detector = new AdaptiveSkinDetector(1, AdaptiveSkinDetector.MorphingMethod.NONE);
                skinDetector = new YCrCbSkinDetector();
            }

I always play with the kinect data in a new thread for speed but you may want to advanced this to a Background worker if you plan to do any more heavy processing so it is better managed.

The thread calls the ProcessBuffer() method you can ignore all the commented code as this is the remanence of the code used to display the depth image. Again I'm using the Marshall copy method to keep things fast but the thing to look for is the Dispatcher.BeginInvoke in WPF that allows the images to be displayed from the Kinect thread. This is required as I'm not processing on the main thread.

        //This takes the byte[] array from the kinect and makes a bitmap from the colour data for us
        byte[] pixeldata = new byte[CF.PixelDataLength];
        CF.CopyPixelDataTo(pixeldata);
        System.Drawing.Bitmap bmap = new System.Drawing.Bitmap(CF.Width, CF.Height, System.Drawing.Imaging.PixelFormat.Format32bppRgb);
        BitmapData bmapdata = bmap.LockBits(new System.Drawing.Rectangle(0, 0, CF.Width, CF.Height), ImageLockMode.WriteOnly, bmap.PixelFormat);
        IntPtr ptr = bmapdata.Scan0;
        Marshal.Copy(pixeldata, 0, ptr, CF.PixelDataLength);
        bmap.UnlockBits(bmapdata);
        //display our colour frame
        currentFrame = new Image<Bgr, Byte>(bmap);


        Image<Gray, Byte> skin2 = skinDetector.DetectSkin(currentFrame, YCrCb_min, YCrCb_max);
        ExtractContourAndHull(skin2);

        DrawAndComputeFingersNum();

        //Display our images using WPF Dispatcher Invoke as this is a sub thread.
        Dispatcher.BeginInvoke((Action)(() =>
            {
                ColorImage.Source = BitmapSourceConvert.ToBitmapSource(currentFrame);
            }), System.Windows.Threading.DispatcherPriority.Render, null);
        Dispatcher.BeginInvoke((Action)(() =>
            {
                SkinImage.Source = BitmapSourceConvert.ToBitmapSource(skin2);
            }), System.Windows.Threading.DispatcherPriority.Render, null);

I hope this helps I will at some point neaten up the code I uploaded,

Cheers

Chris
  • 3,462
  • 20
  • 32
  • Thanks alot for your comment. I still can't figure how to solve it. When I use your code, I get a few errors which I can't seem to solve (probably because of my lack of knowledge). However, here is the code I am using: https://dl.dropboxusercontent.com/u/71972424/codeHandTracker.zip – Arnout Jan 14 '14 at 12:37
  • I got your code to work, but it still gives me the same error. – Arnout Jan 14 '14 at 12:53
  • And if interested, here is the link to the complete solution including the demo. https://dl.dropboxusercontent.com/u/71972424/HandGestureRecognition.rar – Arnout Jan 14 '14 at 13:15
  • Thanks for your updated answer. It is really a help. I am still getting an error I don't understand and can't figure out. I am using the same version as you of emguCV. Didn't edit any DLL's. The error I now get is: https://dl.dropboxusercontent.com/u/71972424/newError1.PNG, so I added opencv_core242.dll to the project like this: https://dl.dropboxusercontent.com/u/71972424/newError2.PNG and set the 'Copy to output directory' to Copy always. But it didn't solve the problem. – Arnout Jan 20 '14 at 08:41
  • http://stackoverflow.com/questions/8028523/unable-to-load-cvextern-in-a-c-sharp-project/8040813#8040813 – Chris Jan 21 '14 at 11:54
  • Thanks. After adding all dll's, it still wouldn't work for some reason. But after a few hours it suddently worked. I think one of the dll's weren't correct. Almost all code runs fine now, but I now get an exception I have never encountered before and once again have no idea how to solve it.. https://dl.dropboxusercontent.com/u/71972424/AccessViolatoinExceptoin.PNG Google didn't give me any solutoins either, I'm sorry to bother you again.. – Arnout Jan 21 '14 at 15:20
  • Hi, this is usually a cross thread access indication what line of code is it occurring on? Cheers – Chris Jan 22 '14 at 14:09
  • Hi, it happens on the following line of code, when entering the loop: #region defects drawing for (int i = 0; i < defects.Total; i++) { System.Drawing.PointF startPoint = new System.Drawing.PointF((float)defectArray[i].StartPoint.X, (float)defectArray[i].StartPoint.Y); The whole loop: https://dl.dropboxusercontent.com/u/71972424/errorMemory.PNG – Arnout Jan 23 '14 at 17:17