0

I am trying to write a test for a windows form app to automate certain UI actions. I learned I can send clicks to elements(windows) that I have handle of. In my app, certain buttons don’t seem to have a handle ID (Spy++ doesn’t find them) But both inspect.exe and VisualUIAVerifyNative.exe can send clicks when selecting elements in their respective tree views. How do I do the same with windows api?

miran80
  • 945
  • 7
  • 22
  • Why are you preemptively rejecting the solution? You already know the solution (UI Automation), so why aren't you using it? – IInspectable Jun 01 '20 at 12:37
  • You *could* send Inputs, but you usually don't need it, since UI Automation allows to [InvokePattern](https://learn.microsoft.com/en-us/dotnet/api/system.windows.automation.invokepattern) and trigger default Elements actions, as clicking a Button, selecting Text, opening and closing Menus etc. So it doesn't really matter whether a Control has a handle. – Jimi Jun 01 '20 at 12:40

1 Answers1

1

Both these apps use UI Automation which is a standard Windows API.

For example, let's say you have a window (a Winform for example) that has a button, then you can click on it using the Invoke Pattern with a code like this, in any other application (like a Console app):

static void Main(string[] args)
{
    // get the app process
    var process = Process.GetProcessesByName("WindowsFormsApp1")[0];

    // get the element corresponding to the main handle (=> the form)
    var element = AutomationElement.FromHandle(process.MainWindowHandle); // needs a reference to UIAutomationClient

    // find the first button in that element
    var button = element.FindFirst(TreeScope.Subtree,
        new PropertyCondition(AutomationElement.ControlTypeProperty, ControlType.Button));  // needs a reference to UIAutomationTypes

    // get the pattern and invoke (=> click)
    button.TryGetCurrentPattern(InvokePattern.Pattern, out var p);
    var pattern = (InvokePattern)p;
    pattern.Invoke();
}

If you own the form, you can even set the AccessibleName button property to something, for example "MyButton", and get it directly by this name, like this:

var button = element.FindFirst(TreeScope.Subtree,
    new PropertyCondition(AutomationElement.NameProperty, "MyButton"));
Simon Mourier
  • 132,049
  • 21
  • 248
  • 298
  • Thank you, Simon. Would you happen to know how to handle program flow when I need to wait undetermined amount of time after one click before sending the next click because a window that I want to click next doesn’t exist yet? Basically, is there a clean await pattern or can I only do it with a timer and retry finding the next button every X seconds? – miran80 Jun 02 '20 at 01:20
  • 1
    @miran80 - not really. One thing for sure is you should never wait one operation for a fixed time because it depends on lots of external parameters. But the wait+retry (possibly with a max timeout) is the usual pattern. Also beware of the TreeScope value. I used SubTree because it's more simple to write, but for big trees, it can take ages (even be infinite). – Simon Mourier Jun 02 '20 at 06:47
  • I noticed that if I tried to recursively parse the tree to get all buttons and their names, it got into infinite loop. But the tree in UIAVerify does not seem to be infinite. Why is that? – miran80 Jun 02 '20 at 10:42
  • 1
    @miran80 - The UI tree is a very live thing, and UIVerify or Insect don't show its entirety, they use lazy loading. Infinite loop can happen because of various legitimate reasons for example if a messagebox is shown into the observed app and no one presses it for some reason. So you should always try to find intermediary elements along the path, using names or ids, of type of control, or a combination of these, etc. – Simon Mourier Jun 02 '20 at 12:38
  • I encountered an app that has a a button that has a name that displays a number that keeps changing. A counter of sorts. The problem is that when I drill down to this name, I keep getting same result as the 1st time I queried it. Even Visual UI Automation Verify tool does not refresh it if I click refresh. Restarting the UIAVerify does updates it but the c# script keeps getting the same name back unchanged as the first time I ran it. Really strange behavior. Any idea what could be wrong? – miran80 Jun 07 '20 at 01:07
  • 1
    You're probably keeping elements that do not correspond to the tree anymore, so you'll need to get them again (from some element in their ascendancy, in the tree above, that is itself still valid), see remarks here: https://stackoverflow.com/questions/7956379/uiautomation-automationelement-force-refresh-current-and-all-descendants – Simon Mourier Jun 07 '20 at 06:34