C# Simulating multitouch with Kinect

Question

I have a Kinect application that I can generate 1-4 distinct screen points (Left/Right hands for up to 2 people) and I would like to be able to send each Point to the application with focus as a multi touch message.

I'm currently using SendInput to send mouse move, mouse down, and mouse up messages, but AFAIK, it doesn't support the WM_TOUCH messages.

Does anyone know of an easy way to send multi touch messages in C#? As a test I would like to be able to use the Kinect in MS Paint, and paint with both hands (as well as all the colors of the wind)

See http://stackoverflow.com/questions/2416748/how-to-simulate-mouse-click-in-c/7121314#7121314 — Liam McInroy, Jul 12 '12 at 20:15
Thanks, but that post talks about mouse messages, I need to send touch messages. Can you send multiple mouse messages? And by that can I have multiple mouses on my screen? — joe_coolish, Jul 12 '12 at 20:30

score 5 · Accepted Answer · edited May 23 '17 at 12:20

5

What you want is to send messages to the window in question. The message you need to compose is the WM_TOUCH message. You can find a very helpful discussion thread on WM_TOUCH here.

Hope this helps!

edited May 23 '17 at 12:20

Community

1
1

answered Jul 13 '12 at 15:04

Ani

10,826
3
27
46

It looks promising! I'll try it out next week and accept if this helps me get it to work :) – joe_coolish Jul 13 '12 at 16:40
Of course - if the message loop of the window you're directing this message to does not use the WM_TOUCH message, nothing will happen :) – Ani Jul 13 '12 at 17:00
yeah, I'm going to have to test for that and then make a primary appendage be the mouse if the application doesn't handle the touch messages – joe_coolish Jul 13 '12 at 17:36

score 0 · Answer 2 · edited May 23 '17 at 12:07

I don't think this would work, unless you did something with saving the coordinates of the x and y's of each persons hand, by putting a canvas down, then an image on top of it, then 4 ellipses, like:, then scaling the position of them to peoples joints, (see Channel 9 for how to do this). Then I would copy the coordinates to a double to then set the pixels of them. Do that like this.

double person1hand1x = Canvas.GetLeft(person1hand1);
double person1hand1y =  Canvas.GetTop(person1hand1);

Then I would change the color of the canvas based on those actions by using the Image control. import the System.Drawing resource into your project, you will need it to set the pixels Then create a Bitmap and set the pixels of it as where the x and y's have been. Do that like this:

        Bitmap b = new Bitmap((int)image1.Width, (int)image1.Height); //set the max height and width

        b.SetPixel(person1hand1x, person1hand1y, person1hand1.Fill); //set the ellipse fill so they can keep track of who drew what


        image1.Source = ToBitmapSource(b); //convert to bitmap source... see https://stackoverflow.com/questions/94456/load-a-wpf-bitmapimage-from-a-system-drawing-bitmap/1470182#1470182 for more details
    }


    /// <summary>
    /// Converts a <see cref="System.Drawing.Bitmap"/> into a WPF <see cref="BitmapSource"/>.
    /// </summary>
    /// <remarks>Uses GDI to do the conversion. Hence the call to the marshalled DeleteObject.
    /// </remarks>
    /// <param name="source">The source bitmap.</param>
    /// <returns>A BitmapSource</returns>
    public static BitmapSource ToBitmapSource(this System.Drawing.Bitmap source)
    {
        BitmapSource bitSrc = null;

        var hBitmap = source.GetHbitmap();

        try
        {
            bitSrc = System.Windows.Interop.Imaging.CreateBitmapSourceFromHBitmap(
                hBitmap,
                IntPtr.Zero,
                Int32Rect.Empty,
                BitmapSizeOptions.FromEmptyOptions());
        }
        catch (Win32Exception)
        {
            bitSrc = null;
        }
        finally
        {
            NativeMethods.DeleteObject(hBitmap);
        }

        return bitSrc;
    }

    internal static class NativeMethods
    {
        [DllImport("gdi32.dll")]
        [return: MarshalAs(UnmanagedType.Bool)]
        internal static extern bool DeleteObject(IntPtr hObject);
    }

Hope this helps! Note: I got the ToBitmapSource from Load a WPF BitmapImage from a System.Drawing.Bitmap

Thanks for the post! I already have the X,Y coords for each appendage and they are in screen coords. I'm actually trying to send multi touch messages to the application with focus. I don't want to recreate a painting program, I want to be able to turn my Kinect into a generic multi touch device. — joe_coolish, Jul 13 '12 at 14:49
@joe_coolish It is impossible to have more than one mouse. If that is what you are saying, also is the painting just an example? Or is that what you are trying to accomplish — Liam McInroy, Jul 13 '12 at 14:50
Just an example. Like how this page uses MSPaint as a test for multi touch monitors: http://aperturering.hubpages.com/hub/MultiTouch-Test. The end goal is to make the Kinect a Multi Touch device — joe_coolish, Jul 13 '12 at 14:54

C# Simulating multitouch with Kinect

2 Answers2

Linked

Related