Visual Studio (C#) - Simulate Win32 Button Press

Question

I need to automate common activities for a few programs, such as Microsoft Word. The testings are not for commercial use, but for regression testing.

Until now I would just move my mouse, and send a left mouse click above the button. However, our program verifies that the button is in "Pressed" state. When the automation presses the button with a mouse click, the button is not pressed (The button does not "sink")

Why is that? How do I cause the automated mouse click to cause the button to be "Pressed" just like a normal mouse click?

In this case, BTW, I am referring to the "Save" button in the quick tool bar.

The code I am using is pretty simple - First I find the object I want to click on, and then this code is used;

public void MouseClick(params string[] args)
{
    AutomationElement automationElement = elementUtils.FindObjectFullPath(args[1], args[4], args[5]);
    string action = args[args.Length - 1];

    if (automationElement == null)
        return;

    FocusApplicationExp(automationElement);

    try
    {
        int x = (int)automationElement.GetClickablePoint().X;
        int y = (int)automationElement.GetClickablePoint().Y;
        MouseMoveExp(x, y);
        MouseAction(action, automationElement);
    }
    catch (Exception)
    {
        Console.WriteLine("Failed to get clickable point for " + automationElement.Current.Name + " - Trying to get coordinates of bounding rectangle");
        tl.AddLine("Failed to get clickable point for " + automationElement.Current.Name + " - Trying to get coordinates of bounding rectangle");
        res.AddLine("Failed to get clickable point for " + automationElement.Current.Name + " - Trying to get coordinates of bounding rectangle");

        int x = (int)(automationElement.Current.BoundingRectangle.Right - automationElement.Current.BoundingRectangle.Width / 2);
        int y = (int)(automationElement.Current.BoundingRectangle.Bottom - automationElement.Current.BoundingRectangle.Height / 2);
        MouseMoveExp(x, y);
        MouseAction(action, automationElement);
        return;
    }
}

private void MouseAction(string action, AutomationElement automationElement)
{
    switch (action)
    {
        case "Left":
            Console.WriteLine("Left clicking on: " + automationElement.Current.Name);
            tl.AddLine("Left clicking on: " + automationElement.Current.Name);
            AutomationUtility.DoMouseClick();
            res.AddLine("Left clicked on: " + automationElement.Current.Name);
            break;
        case "Right":
            Console.WriteLine("Right clicking on: " + automationElement.Current.Name);
            tl.AddLine("Right clicking on: " + automationElement.Current.Name);
            AutomationUtility.DoMouseRightClick();
            res.AddLine("Right clicked on: " + automationElement.Current.Name);
            break;
        case "Double":
            Console.WriteLine("Double clicking on: " + automationElement.Current.Name);
            tl.AddLine("Double clicking on: " + automationElement.Current.Name);
            AutomationUtility.DoMouseDoubleClick();
            res.AddLine("Double clicked on: " + automationElement.Current.Name);
            break;
         default:
            break;
    }
}

public static void DoMouseClick()
{
    Mouse.Click(MouseButton.Left);
}

And I did try a few variations, such as giving some delay between button down and up, in case the event happened but too quick. But the button never enters "Pressed" state even if I keep the button down (in the automation, of course)

Just to be more clear - The automation DOES work, it DOES click the button, and it DOES cause word to save. The issue here is to cause the button to be pressed visually (and program-wise, change it's state).

I am not sure if it means anything, but the system that identifies the button states and events is "UX", and the Win32-based identifier DOES find the "pressed" state somehow.

EDIT: I have been reading up on it for the past few days and things are becoming clearer. The automation I use is based on UIAutomation, but the button state "Pressed" is a MSAA state that did not move onto the UIAutomation generation of UI objects, so UIAutomation-based automation does not invoke this state. In other words, I need to find a way to use UIAutomation code to send MSAA signals to UIAutomation elements (That contain the IAccessible interface). Problem is, it is only supported the other way around (Send UIAutomation signals masked as MSAA signals to MSAA elements, which does not support MSAA-Only signals, such as "Enter state Pressed")

As posted you'd never see it, the machine just clicks too fast. We can't see the alternative you tried. This should not matter for UI testing, all that matters is that *Word* sees the mouse click. — Hans Passant, Jan 02 '14 at 14:20
Problem is, our program (not the automation) collects events, and using them to know if an action was made. In this case, Mouse click on button with name "Save" and parent "Quick Tool Bar", object state: Pressed, Focusable. So I need the automated mouseclick to work like a human's. — Adam Cohen, Jan 02 '14 at 14:24

score 1 · Answer 1 · answered Jan 02 '14 at 14:10

I don't know if you use this way but it might help on the click part

public void ClickOnPoint(IntPtr wndHandle )
{
//for handle you can use (this.Handle)
//you can assign point directly or recalibrate as you did already 
Point buttonPoint(100, 500)
// store old pos if needed
POINT oldPoint;
GetCursorPos(out oldPoint);
// get screen coordinates
ClientToScreen(wndHandle, ref buttonPoint);
// set cursor on coords, and press mouse
SetCursorPos(buttonPoint.X, buttonPoint.Y);
//you can change the timing
mouse_event(0x00000002, 0, 0, 0, UIntPtr.Zero); // left mouse button down
mouse_event(0x00000004, 0, 0, 0, UIntPtr.Zero); // left mouse button up
// return mouse 
SetCursorPos(oldPoint.X, oldPoint.Y);
}

score 1 · Answer 2 · edited May 23 '17 at 12:12

1

Maybe check out using SendInput.

In the post below I posted a few classes for using SendInput.

How to move the cursor or simulate clicks for other applications?

NOTE: I could see the save button go orange if I put in a delay. I did not see it go orange without a delay.

  public static void LeftClick()
  {
     DoMouse( NativeMethods.MOUSEEVENTF.LEFTDOWN, new Point( 0, 0 ) );
     System.Threading.Thread.Sleep( 200 );
     DoMouse( NativeMethods.MOUSEEVENTF.LEFTUP, new Point( 0, 0 ) );
  }

edited May 23 '17 at 12:12

Community

1
1

answered Jan 02 '14 at 14:23

Derek

7,615
5
33
58

I tried it. It works in a sense that it sends the input just right, but still does not cause the button to change state. Thank you anyway! – Adam Cohen Jan 02 '14 at 14:45
No problem. Thought it was worth a try. :) – Derek Jan 02 '14 at 14:50
I've updated the answer.. using SendInput with a delay seems to make the button go orange for a bit. – Derek Jan 02 '14 at 15:07

score 0 · Accepted Answer · answered Jan 06 '14 at 08:46

Well I did not find the answer I was looking for, but this seems to work anyway;

 [System.Runtime.InteropServices.DllImport("user32.dll")]
    public static extern void mouse_event(int dwFlags, int dx, int dy, int cButtons, int dwExtraInfo);

    public const int MOUSEEVENTF_LEFTDOWN = 0x02;
    public const int MOUSEEVENTF_LEFTUP = 0x04;
    public const int MOUSEEVENTF_RIGHTDOWN = 0x08;
    public const int MOUSEEVENTF_RIGHTUP = 0x10;

    //This simulates a left mouse click
    public static void LeftMouseClick(int xpos, int ypos)
    {
        mouse_event(MOUSEEVENTF_LEFTDOWN, xpos, ypos, 0, 0);
        Thread.Sleep(100);
        mouse_event(MOUSEEVENTF_LEFTUP, xpos, ypos, 0, 0);
    }

    public static void RightMouseClick(int xpos, int ypos)
    {
        mouse_event(MOUSEEVENTF_RIGHTDOWN, xpos, ypos, 0, 0);
        Thread.Sleep(100);
        mouse_event(MOUSEEVENTF_RIGHTUP, xpos, ypos, 0, 0);
    }

I am not sure how it is different Derek's or Za7ef's answers, since I tried them both. I have changed the code a few times so it might have had a different reason. Thank you anyways!

Visual Studio (C#) - Simulate Win32 Button Press

3 Answers3