how to control mouse pointer with head movement using emguCV C#?

Question

I am developing a system just like Camera mouse or other face control mouse, I have implemented all the functionality, the mouse pointer is also moving well, but I want to create the movement smooth just like the mouse control the pointer. the code I am using is:

        if (startButton == true)
        {
            try
            {
                cap = new Capture();
                pictureBox1.Image = cap.QueryFrame().ToImage<Bgr, Byte>().Bitmap;
            }
            catch (Exception exp)
            {
                MessageBox.Show("Error:" + exp);
            }
            _cascadeClassifier = new CascadeClassifier(Application.StartupPath + "/haarcascade_frontalface_default.xml");
            eye_cascadeClassifier = new CascadeClassifier(Application.StartupPath + "/haarcascade_eye.xml");

            timer1.Start();
        }

    private void timer1_Tick(object sender, EventArgs e)
    {
        using (var imageFrame = cap.QueryFrame().ToImage<Bgr, Byte>().Flip(FlipType.Horizontal))
        {
            if (imageFrame != null)
            {
                var grayframe = imageFrame.Convert<Gray, byte>();

                var faces = _cascadeClassifier.DetectMultiScale(grayframe, 1.1, 10, Size.Empty); //the actual face detection happens here

                foreach (var face in faces)
                {
                    if(Configure.FaceBoxCheck==true)
                    imageFrame.Draw(face, new Bgr(Color.LightGreen), 2); //the detected face(s) is highlighted here using a box that is drawn around it/them
                    Int32 yCoordStartSearchEyes = face.Top + (face.Height * 3 / 11);
                    Point startingPointSearchEyes = new Point(face.X, yCoordStartSearchEyes);
                    Size searchEyesAreaSize = new Size(face.Width, (face.Height * 3 / 11));
                    Rectangle possibleROI_eyes = new Rectangle(startingPointSearchEyes, searchEyesAreaSize);

                    int widthNav = (imageFrame.Width / 11 * 3);
                    int heightNav = (imageFrame.Height / 11 * 3);
                    Rectangle nav = new Rectangle(new Point(imageFrame.Width / 2 - widthNav / 2, imageFrame.Height / 2 - heightNav / 2), new Size(widthNav, heightNav));
                    imageFrame.Draw(nav, new Bgr(Color.Lavender), 3);
                    Point cursor = new Point(face.X + searchEyesAreaSize.Width / 2, yCoordStartSearchEyes + searchEyesAreaSize.Height / 2);
                    grayframe.ROI = possibleROI_eyes;
                    var eyes = eye_cascadeClassifier.DetectMultiScale(grayframe, 2.15, 3, Size.Empty);

                    foreach (var eye in eyes)
                    {
                      //imageFrame.Draw(eye, new Bgr(Color.Red), 2);
                        if(Configure.EyeBoxCheck==true)
                        imageFrame.Draw(possibleROI_eyes, new Bgr(Color.DarkGreen), 2);
                        if (nav.Left < cursor.X && cursor.X < (nav.Left + nav.Width) && nav.Top < cursor.Y && cursor.Y < nav.Top + nav.Height)
                        {
                            LineSegment2D CursorDraw = new LineSegment2D(cursor, new Point(cursor.X, cursor.Y + 1));

                            imageFrame.Draw(CursorDraw, new Bgr(Color.White), 3);
                            //we compute new cursor coordinate using a simple scale based on frame width and height
                            int xCoord = (imageFrame.Width * (cursor.X - nav.Left)) / nav.Width;
                            int yCoord = (imageFrame.Height * (cursor.Y - nav.Top)) / nav.Height;
                            //We set our new cursor position
                            Cursor.Position = new Point(xCoord * 2, yCoord *2);
                    }
                }
            }

Answer 1

Ok, I'm sure there are a lot of other better ways, but this is a quick&dirty way of moving cursor position in a "Smooth" way from point a to point b. Of course this implementation can and should be optimized using a different thread instead of using Application.DoEvents() to avoid blocking the UI thread, but i hope this gets you on the track. First, how you should use it. Instead of:

Cursor.Position = new Point(xCoord * 2, yCoord *2);

You should do this:

MoveCursorSmooth(Cursor.Position, new Point(xCoord * 2, yCoord *2));

Now, the implementation of MoveCursorSmooth:

    private void MoveCursorSmooth(Point a, Point b)
    {
        var step = 5;
        var left = Math.Min(a.X, b.X);
        var right = Math.Max(a.X, b.X);
        int width = right - left;

        var top = a.Y;
        var bottom = b.Y;
        int height = bottom - top;

        if (width > height)
        {
            double slope = (double)height / (double)width;
            if (a.X <= b.X) 
                for (int x = 1; x < width; ++x) 
                {
                    Cursor.Position = new Point((left + x), (a.Y + ((int)(slope * x + 0.5))));
                    System.Threading.Thread.Sleep(step);
                    Application.DoEvents();
                }
            else
                for (int x = 1; x < width; ++x) // xOffset
                {
                    Cursor.Position = new Point((right - x), (a.Y + ((int)(slope * x + 0.5))));
                    System.Threading.Thread.Sleep(step);
                    Application.DoEvents();
                }
        }
        else
        {

            double slope = (double)width / (double)height;
            if (a.X <= b.X)
            {
                for (int y = 1; y < height; ++y)
                {
                    Cursor.Position = new Point((a.X + ((int)(slope * y + 0.5))), (top + y));
                    System.Threading.Thread.Sleep(step);
                    Application.DoEvents();
                }
            }
            else
            {
                for (int y = 1; y < height; ++y)
                {
                    Cursor.Position = new Point((b.X + ((int)(slope * y + 0.5))), (bottom - y));
                    System.Threading.Thread.Sleep(step);
                    Application.DoEvents();
                }
            }
        }
    }

This method is based on this answer