SharpGL & Kinect - Rendering ColorImageFrame Bitmap to OpenGL Texture

Question

I want to draw a simple 2D quad with a texture from the Kinect's video input but my code isn't working.

Here is my ColorFrameReady event:

    void Sensor_ColorFrameReady(object sender, ColorImageFrameReadyEventArgs e)
    {
        ColorImageFrame cif = e.OpenColorImageFrame();

        if (cif != null)
        {
            middleBitmapSource = cif.ToBitmapSource();
            cif.Dispose();
        }
    }

This is my OpenGL draw method:

    private void OpenGLControl_OpenGLDraw(object sender, OpenGLEventArgs args)
    {
        // Get the OpenGL instance being pushed to us.
        gl = args.OpenGL;

        // Clear the colour and depth buffers.
        gl.Clear(OpenGL.GL_COLOR_BUFFER_BIT | OpenGL.GL_DEPTH_BUFFER_BIT);

        // Enable textures
        gl.Enable(OpenGL.GL_TEXTURE_2D);

        // Reset the modelview matrix.
        gl.LoadIdentity();

        // Load textures
        if (middleBitmapSource != null)
        {
            middleBitmap = getBitmap(middleBitmapSource, quadMiddleWidth, quadMiddleHeight);
            //middleBitmap = new System.Drawing.Bitmap("C:\\Users\\Bobble\\Pictures\\Crate.bmp"); // When I use this texture, it works fine
            gl.GenTextures(1, textures);
            gl.BindTexture(OpenGL.GL_TEXTURE_2D, textures[0]);
            gl.TexImage2D(OpenGL.GL_TEXTURE_2D, 0, 3, middleBitmap.Width, middleBitmap.Height, 0, OpenGL.GL_BGR, OpenGL.GL_UNSIGNED_BYTE,
                middleBitmap.LockBits(new System.Drawing.Rectangle(0, 0, middleBitmap.Width, middleBitmap.Height),
                System.Drawing.Imaging.ImageLockMode.ReadOnly, System.Drawing.Imaging.PixelFormat.Format24bppRgb).Scan0);

            gl.TexParameter(OpenGL.GL_TEXTURE_2D, OpenGL.GL_TEXTURE_MIN_FILTER, OpenGL.GL_LINEAR);
            gl.TexParameter(OpenGL.GL_TEXTURE_2D, OpenGL.GL_TEXTURE_MAG_FILTER, OpenGL.GL_LINEAR);
        }

        // Move back a bit
        gl.Translate(0.0f, 0.0f, -25.0f);

        gl.BindTexture(OpenGL.GL_TEXTURE_2D, textures[0]);

        // Draw a quad.
        gl.Begin(OpenGL.GL_QUADS);

            // Draw centre quad
            gl.TexCoord(0.0f, 0.0f); gl.Vertex(-(quadMiddleWidth / 2), quadMiddleHeight / 2, quadDepthFar);
            gl.TexCoord(0.0f, 1.0f); gl.Vertex(quadMiddleWidth / 2, quadMiddleHeight / 2, quadDepthFar);
            gl.TexCoord(1.0f, 1.0f); gl.Vertex(quadMiddleWidth / 2, -(quadMiddleHeight / 2), quadDepthFar);
            gl.TexCoord(1.0f, 0.0f); gl.Vertex(-(quadMiddleWidth / 2), -(quadMiddleHeight / 2), quadDepthFar);

        gl.End();

        gl.Flush();
    }

And here is a helper method which converts a BitmapSource to a Bitmap of a given size:

    private System.Drawing.Bitmap getBitmap(BitmapSource bitmapSource, double rectangleWidth, double rectangleHeight)
    {
        double newWidthRatio = rectangleWidth / (double)bitmapSource.PixelWidth;
        double newHeightRatio = ((rectangleWidth * bitmapSource.PixelHeight) / (double)bitmapSource.PixelWidth) / (double)bitmapSource.PixelHeight;

        BitmapSource transformedBitmapSource = new TransformedBitmap(bitmapSource, new ScaleTransform(newWidthRatio, newHeightRatio));

        int width = transformedBitmapSource.PixelWidth;
        int height = transformedBitmapSource.PixelHeight;
        //int stride = width * ((transformedBitmapSource.Format.BitsPerPixel + 7) / 8);
        int stride = ((width * transformedBitmapSource.Format.BitsPerPixel + 31) & ~31) / 8; // See: http://stackoverflow.com/questions/1983781/why-does-bitmapsource-create-throw-an-argumentexception/1983886#1983886

        byte[] bits = new byte[height * stride];

        transformedBitmapSource.CopyPixels(bits, stride, 0);

        unsafe
        {
            fixed (byte* pBits = bits)
            {
                IntPtr ptr = new IntPtr(pBits);

                System.Drawing.Bitmap bitmap = new System.Drawing.Bitmap(
                    width, height, stride, System.Drawing.Imaging.PixelFormat.Format32bppArgb, ptr);

                return bitmap;
            }
        }
    }

The quad renders but it is just white, no texture on it.

score 0 · Accepted Answer · answered May 04 '12 at 13:47

0

This turned out to be an issue with SharpGL - the OpenGL library I was using.

http://sharpgl.codeplex.com/discussions/348278 - discussion on CodePlex which led to the issue being resolved by the SharpGL developers.

answered May 04 '12 at 13:47

bobble14988

1,749
5
26
38

Is ToBitmapSource part of the SDK or is that your function? I'm new to the kinect SDK, and not able to find it. – Frank Schwieterman Mar 24 '13 at 01:43
This is a very old project so bear with me. The it looks like the ToBitmapSource function is part of the SDK but I can't see it in the documentation. PM me via bobble14988@gmail.com and I'll see if I can dig you out some code. – bobble14988 Apr 03 '13 at 15:51
Thanks bobble, later I found something here: http://stackoverflow.com/questions/10848190/convert-kinect-colorframe-to-bitmap – Frank Schwieterman Apr 03 '13 at 16:51

score 0 · Answer 2 · edited Jun 20 '20 at 09:12

0

I was having a white picture too, but when I changed the input of the function TexImage2D to use width and heights which are multiples of 2, it worked. According to the documentation:

The width of the texture image (must be a power of 2, e.g 64).

The height of the texture image (must be a power of 2, e.g 32).

edited Jun 20 '20 at 09:12

Community

1
1

answered May 12 '13 at 23:20

Jean-Philippe Bergeron

26
2

SharpGL & Kinect - Rendering ColorImageFrame Bitmap to OpenGL Texture

2 Answers2