Virtual Keyboard (Swype like Keyboard) Windows Forms Application C#

Question

I am trying to create a swype like keyboard in Windows Forms using c#

I have two problems: a. I am not able to reproduce the finger swipe action. b. I am not able to recognize the letters under different keys.

For the first problem:

I used the Graphics Class and the Pen class and used it like this :

    bool draw;
    Graphics g;
    Pen p;
    int prevX;
    int prevY;

    private void Form1_Load(object sender, EventArgs e)
    {
        draw = false;
        g = CreateGraphics();
        p = new Pen(Color.Black, 2);
    }

    private void Form1_MouseDown(object sender, MouseEventArgs e)
    {
        draw = true;
        prevX = e.X;
        prevY = e.Y;
    }

    private void Form1_MouseUp(object sender, MouseEventArgs e)
    {
        draw = false;
    }

    private void Form1_MouseMove(object sender, MouseEventArgs e)
    {
        if (draw)
        {
            g.DrawLine(p, prevX, prevY, e.X, e.Y);
            prevX = e.X;
            prevY = e.Y;
        }
    }

But the problem here is that it starts drawing from the Left-top corner of the screen and not from the position of the button. It also does't draw over the buttons. Any control that is in the path overlaps the line.

For the second Issue:

a. I used the mouse down event to set a boolean value "Allow" as true and then when the mouse moves I tried getting the text of the button. Just as in the above picture. I tried to start from letter "S" and as I move over other letters only the letter "S" keeps recording. The Mouse Enter or Mouse Hover event of the other methods do not even occur.

b. I also tried using dragdrop event but it doesn't work too. I am not sure but I am assuming that as buttons are not draggable objects the dragdrop event doesn't work.

I don't understand or am confused how to achieve my goal.

I assume when you start drawing that you get a line from the top left corner to the mouse position? This is probably because your prevx/prevy variables are default (0), in this case you want to defer drawing until the second mouse move event which should give you the right result. I can't comment on the rest of the question though — Charleh, Sep 30 '17 at 13:56
This is pretty fundamental. You cannot use Button controls, they clip anything you draw and have their own mouse handling. You have to make them *look* like buttons with the Paint event. — Hans Passant, Sep 30 '17 at 13:59
@HansPassant Is there a way to detect the letters while doing the motion? — Akhil, Sep 30 '17 at 14:42
You first need a hittest method that can select the correct letter from a normal mouse click. Once you have that, doing it "in motion" is not going to be an obstacle. — Hans Passant, Sep 30 '17 at 14:46
@Akhil Take a look at *Solution 2* in [this post](https://stackoverflow.com/a/40209045/3110834). — Reza Aghaei, Oct 08 '17 at 02:38

Reza Aghaei · Accepted Answer · 2017-10-09T01:59:29.493

You can create a custom control and handle its drawing yourself.

But if you are going to use Button controls and want to draw over those buttons, you can put a transparent panel as overlay on those buttons and draw the swipe path over the transparent panel:

Transparent Panel

using System.Windows.Forms;
public class TransparentPanel : Panel
{
    const int WS_EX_TRANSPARENT = 0x20;
    protected override CreateParams CreateParams
    {
        get
        {
            CreateParams cp = base.CreateParams;
            cp.ExStyle = cp.ExStyle | WS_EX_TRANSPARENT;
            return cp;
        }
    }
    protected override void OnPaintBackground(PaintEventArgs e)
    {
    }
}

Sample Form

This is just an example to demonstrate the usage of transparent control. To do so, it's enough to create a form and replace it's contents with following code:

TransparentPanel overlay;
TableLayoutPanel table;
List<Point> points = new List<Point>();
List<String> keys = new List<string>();
public Form1()
{
    overlay = new TransparentPanel() { Dock = DockStyle.Fill };
    this.Controls.Add(overlay);

    table = new TableLayoutPanel()
    { ColumnCount = 6, RowCount = 4, Dock = DockStyle.Fill };
    this.Controls.Add(table);

    var list = Enumerable.Range('A', 'Z' - 'A' + 1)
        .Select(x => ((char)x).ToString()).ToList();

    list.ForEach(x => table.Controls.Add(new Button()
    { Text = x, Width = 32, Height = 32 }));

    overlay.MouseDown += OverlayMouseDown;
    overlay.MouseMove += OverlayMouseMove;
    overlay.MouseUp += OverlayMouseUp;
    overlay.Paint += OverlayPaint;
}
void OverlayMouseMove(object sender, MouseEventArgs e)
{
    if (e.Button == MouseButtons.Left)
        ProccessPoint(e.Location);
}
void OverlayMouseDown(object sender, MouseEventArgs e)
{
    Clear();
    ProccessPoint(e.Location);
}
void OverlayMouseUp(object sender, MouseEventArgs e)
{
    if (points.Count > 0)
        MessageBox.Show(string.Join(",", keys));
    Clear();
}
void OverlayPaint(object sender, PaintEventArgs e)
{
    if (points.Count >= 3)
        e.Graphics.DrawCurve(Pens.Red, points.ToArray());
}
void ProccessPoint(Point p)
{
    points.Add(p);
    var c = table.Controls.Cast<Control>()
        .Where(x => table.RectangleToScreen(x.Bounds)
        .Contains(overlay.PointToScreen(p))).FirstOrDefault();
    if ((c != null) && (keys.Count == 0 || keys[keys.Count - 1] != c.Text))
        keys.Add(c.Text);
    overlay.Invalidate();
}
void Clear()
{
    keys.Clear();
    points.Clear();
    this.Refresh();
}

Virtual Keyboard (Swype like Keyboard) Windows Forms Application C#

1 Answers1

Linked