2

I'm trying to automate a sub window (Dialog with ClassName is #32770) by using UI Automation C# API.

The target sub window shows up in Spy++ and Inspect.exe, but doesn't show in VisualUIAVerifyNative. It also could not be accessed by either AutomationElement.FindFirst or TreeWalker navigation.

I checked the tree in VisualUIAVerifyNative, found it was quite different from the tree in Spy++ and Inspect.exe.

Interesting things:

  1. My C# code seems behave exactly like VisualUIAVerifyNative.
  2. I'm able to use AutomationElement.FromHandle to bind the HWND of the Dialog

Do you know why this happens?

Here's the detail of The Dialog from Insepct.exe

How found:  Selected from tree...
Name:   "V6"
ControlType:    UIA_PaneControlTypeId (0xC371)
LocalizedControlType:   "Dialog"
BoundingRectangle:  {l:-47997 t:-47997 r:-46107 b:-47553}
IsEnabled:  true
HasKeyboardFocus:   false
ProcessId:  15496
RuntimeId:  [2A.140BD8]
FrameworkId:    "Win32"
ClassName:  "#32770"
NativeWindowHandle: 0x140BD8
IsControlElement:   false
IsContentElement:   false
ProviderDescription:    "[pid:10556,hwnd:0x140BD8 Main:Microsoft: Container Proxy (unmanaged:uiautomationcore.dll); Hwnd(parent link):Microsoft: HWND Proxy (unmanaged:uiautomationcore.dll)]"
LegacyIAccessible.ChildId:  0
LegacyIAccessible.Name: "V6"
LegacyIAccessible.Role: Dialog (0x12)
LegacyIAccessible.State:     (0x100000)
IsAnnotationPatternAvailable:   false
IsDragPatternAvailable: false
IsDockPatternAvailable: false
IsDropTargetPatternAvailable:   false
IsExpandCollapsePatternAvailable:   false
IsGridItemPatternAvailable: false
IsGridPatternAvailable: false
IsInvokePatternAvailable:   false
IsItemContainerPatternAvailable:    false
IsLegacyIAccessiblePatternAvailable:    true
IsMultipleViewPatternAvailable: false
IsObjectModelPatternAvailable:  false
IsRangeValuePatternAvailable:   false
IsScrollItemPatternAvailable:   false
IsScrollPatternAvailable:   false
IsSelectionItemPatternAvailable:    false
IsSelectionPatternAvailable:    false
IsSpreadsheetItemPatternAvailable:  false
IsSpreadsheetPatternAvailable:  false
IsStylesPatternAvailable:   false
IsSynchronizedInputPatternAvailable:    false
IsTableItemPatternAvailable:    false
IsTablePatternAvailable:    false
IsTextChildPatternAvailable:    false
IsTextEditPatternAvailable: false
IsTextPatternAvailable: false
IsTextPattern2Available:    false
IsTogglePatternAvailable:   false
IsTransformPatternAvailable:    false
IsTransform2PatternAvailable:   false
IsValuePatternAvailable:    false
IsVirtualizedItemPatternAvailable:  false
IsWindowPatternAvailable:   false
...

Just found this https://stackoverflow.com/a/14187618/1633272.

I tried the TreeWalker.RawViewWalker, it worked. However TreeWalker.ControlViewWalker didn't.

Community
  • 1
  • 1
user1633272
  • 2,007
  • 5
  • 25
  • 48
  • As far as I can tell, the root cause is that `IsControlElement` and/or `IsContentElement` are `false`. Just like @VasilyRyabov and others wrote - I'm pretty sure that it's a bug within `UIAutomationCore.dll`. – itsho Jan 15 '20 at 14:33

3 Answers3

3

It's a known difference (.NET specific bug) between native COM API and managed API. Here is an example how to use UIA through COM objects for most popular languages including C#.

Vasily Ryabov
  • 9,386
  • 6
  • 25
  • 78
  • It was too long, I could't recall what the problem is. I end up with native C++ to this kind of work. – user1633272 Sep 11 '17 at 07:00
  • If you've chosen C++, then why not Python? pywinauto already wraps this native COM interface with higher level stuff. – Vasily Ryabov Sep 11 '17 at 07:23
  • That's a close source commercial software, anyway, most work is done. I'm pretty happy with C++. – user1633272 Sep 11 '17 at 07:25
  • Anyway, I think UIA is not practical for several cases. Use native C++ if you can, things will be easier for complex work. – user1633272 Sep 11 '17 at 07:30
  • Sure. pywinauto can do the job by UIA or Win32 native stuff. Easy to switch between each other. I had some experience with debugging C++ GUI tests 8 years ago for a half of year. It was a nightmare for me. The only case I've ever used debug&continue feature. – Vasily Ryabov Sep 11 '17 at 10:09
2

Today, I solved a similar issue I had last year which I didn't have time to revisit until now.

My problem was, the modal dialog of an application can't be found with UIAutomation interface (I tried native COM calls in C, and other methods with C# with all possible methods of UIA, to no success), and inspect.exe showed it quite easily, very similar to yours, maybe same.

So, when I tried finding the same dialog with FindWindowEx method, it worked quite well. One problem that surprised me was, the dialog was shown under it's application's main window in Inspect's tree hierarchy, but FindWindowEx was only successful when I searched for the dialog as desktop child - which occurs for some cases I believe, for example if you don't properly set the ShowDialog() receiver's parent, it is attached to Desktop.

This might answer some other questions I had about inspect.exe's behaviour. Speculating about it, I think it might be using winapi calls to find windows, and generating a tree hierarchy that's slightly different than the actual, thus leading us poor users baffled for a long time. Bad design.

And here is some code, if you still want to try it after a year, or to help someone else. To find a window under desktop:

First define this:

 [DllImport("user32.dll", SetLastError = true)]
    public static extern IntPtr FindWindowEx(IntPtr parentHandle, IntPtr childAfter, string className, string windowTitle);

Then call:

IntPtr w = FindWindowEx(IntPtr.Zero, IntPtr.Zero, (string)null, "V6 - or your dialog's name");

or to find it as child of a top level window (find it's handle from IUIAutomation.CurrentNativeWindowHandle, let's say set to var handle):

IntPtr w = FindWindowEx(handle, IntPtr.Zero, (string)null, "V6 - or your dialog's name");

Then you can try the w with IUIAutomation.ElementFromHandle().

AlbusMPiroglu
  • 645
  • 3
  • 13
0

Although question asked is older but I had similar issue and solved it as below. As IsControlElement property of the dialog is false so you could have access this dialog using RawViewWalker. Here is the sample.

            IEnumerable<AutomationElement> controlElements = GetChildren(parentAutomationElement);
            AutomationElement automationElement = null;
            foreach (var element in controlElements)
            {
                if (element.Current.ClassName == "#32770")
                {
                    automationElement = element;
                    break;
                }
            }`
    private List<AutomationElement> GetChildren(AutomationElement parent)
    {
        if (parent == null)
        {
            // null parameter
            throw new ArgumentException();
        }

        IEnumerable<AutomationElement> collection = parent.FindInRawView();

        if (collection != null)
        {
            List<AutomationElement> result = new List<AutomationElement>(collection.Cast<AutomationElement>());
            return result;
        }
        else
        {
            // some error occured
            return null;
        }
    }
public static IEnumerable<AutomationElement> FindInRawView(this AutomationElement root)
    {
        TreeWalker rawViewWalker = TreeWalker.RawViewWalker;
        Queue<AutomationElement> queue = new Queue<AutomationElement>();
        queue.Enqueue(root);
        while (queue.Count > 0)
        {
            var element = queue.Dequeue();
            yield return element;

            var sibling = rawViewWalker.GetNextSibling(element);
            if (sibling != null)
            {
                queue.Enqueue(sibling);
            }

            var child = rawViewWalker.GetFirstChild(element);
            if (child != null)
            {
                queue.Enqueue(child);
            }
        }
    }

Make sure to put the FindInRawView Extension Method in a static class, else it will show error.

Farrukh Niaz
  • 21
  • 3