Chrome .Net Ui Automation - Automatic Sign on to Browser - AutomationElement Invoke appears to invoke with no result

Question

I would like to automatically sign into Chrome on Windows, the Selenium part is done but now there is a Window that needs to be automated. The invocation of the Link data push button. It appears I find the right element with my code and it does attempt an invoke but to no avail.

Below snippet is from WPF :

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Windows;
using System.Windows.Controls;
using System.Windows.Data;
using System.Windows.Documents;
using System.Windows.Input;
using System.Windows.Media;
using System.Windows.Media.Imaging;
using System.Windows.Navigation;
using System.Windows.Shapes;
using System.Windows.Automation;
using System.Diagnostics;
using System.Runtime.InteropServices;

namespace Chrome_WPF_SSO
{

    /// <summary>
    /// Interaction logic for MainWindow.xaml
    /// </summary>
    public partial class MainWindow : Window
    {

        public InvokePattern GetInvokePattern(AutomationElement element)
        {
            return element.GetCurrentPattern(InvokePattern.Pattern) as InvokePattern;
        }

        public MainWindow()
        {
            InitializeComponent();

            //1
            var Chrome = Process.GetProcessesByName("chrome");
            if (Chrome.Length <= 0) { }

            else
            {
                foreach (var cp in Chrome)
                {
                    if (!(cp.MainWindowHandle == IntPtr.Zero))
                    {

                        //To find the tabs we first need to locate something reliable - the 'New Tab' button
                        AutomationElement rootElement = AutomationElement.FromHandle(cp.MainWindowHandle);
                        System.Windows.Automation.Condition condNewTab = new PropertyCondition(AutomationElement.NameProperty, "Google Chrome");


                        System.Windows.Automation.Condition condCustom = new PropertyCondition(AutomationElement.NameProperty, "Link your Chrome data to this account?");
                        AutomationElementCollection allNames = rootElement.FindAll(TreeScope.Descendants, condCustom);
                        TreeWalker tWalker2 = TreeWalker.ControlViewWalker;

                        System.Windows.Automation.Condition LinkData = new PropertyCondition(AutomationElement.NameProperty, "Link data");
                        AutomationElementCollection elementscoll = rootElement.FindAll(TreeScope.Descendants, LinkData);

                        foreach (AutomationElement a in elementscoll)
                        {

                            if (a.Current.LocalizedControlType.ToString() == "button")
                            {
                                try
                                {
                                    ((InvokePattern)a.GetCurrentPattern(InvokePattern.Pattern)).Invoke();


                                }

                                catch (Exception ex)
                                {
                                    MessageBox.Show(ex.Message);
                                }

                            }

                        }
                    }


                }


            }
        }
    }
}

From the Win SDK inspect.exe tool:

        How found:  Selected from tree...
Name:   "Link data"
ControlType:    UIA_ButtonControlTypeId (0xC350)
LocalizedControlType:   "button"
BoundingRectangle:  {l:740 t:271 r:816 b:304}
IsEnabled:  true
IsOffscreen:    false
IsKeyboardFocusable:    true
HasKeyboardFocus:   false
AcceleratorKey: [Not supported]
AccessKey:  ""
ProcessId:  4832
RuntimeId:  [Not supported]
AutomationId:   [Not supported]
FrameworkId:    [Not supported]
ClassName:  [Not supported]
NativeWindowHandle: [Not supported]
IsControlElement:   [Not supported]
IsContentElement:   [Not supported]
ProviderDescription:    "[pid:4832,hwnd:0x0 Main(parent link):Microsoft: MSAA Proxy (unmanaged:uiautomationcore.dll)]"
IsPeripheral:   [Not supported]
AriaRole:   [Not supported]
AriaProperties: [Not supported]
LiveSettingProperty:    [Not supported]
IsPassword: false
ItemStatus: [Not supported]
ItemType:   [Not supported]
IsRequiredForForm:  [Not supported]
IsDataValidForForm: [Not supported]
HelpText:   "LabelButton"
LabeledBy:  [Not supported]
DescribedBy:    [Not supported]
ControllerForProperty:  [Not supported]
FlowsTo:    [Not supported]
FlowsFrom:  [Not supported]
ClickablePoint: [Not supported]
Culture:    [Not supported]
Orientation:    [Not supported]
OptimizeForVisualContent:   [Not supported]
Annotation.AnnotationAuthor:    [Not supported]
Annotation.AnnotationTypeId:    [Not supported]
Annotation.Author:  [Not supported]
Annotation.DateTime:    [Not supported]
Annotation.Target:  [Not supported]
Drag.DropEffect:    [Not supported]
Drag.DropEffects:   [Not supported]
Drag.GrabbedItems:  [Not supported]
Drag.IsGrabbed: [Not supported]
Dock.DockPosition:  [Not supported]
DropTarget.DropTargetEffect:    [Not supported]
DropTarget.DropTargetEffects:   [Not supported]
ExpandCollapse.ExpandCollapseState: [Not supported]
Grid.ColumnCount:   [Not supported]
Grid.RowCount:  [Not supported]
GridItem.Column:    [Not supported]
GridItem.ColumnSpan:    [Not supported]
GridItem.ContainingGrid:    [Not supported]
GridItem.Row:   [Not supported]
GridItem.RowSpan:   [Not supported]
LegacyIAccessible.ChildId:  0
LegacyIAccessible.DefaultAction:    ""
LegacyIAccessible.Description:  ""
LegacyIAccessible.Help: "LabelButton"
LegacyIAccessible.KeyboardShortcut: ""
LegacyIAccessible.Name: "Link data"
LegacyIAccessible.Role: push button (0x2B)
LegacyIAccessible.State:    focusable (0x100000)
LegacyIAccessible.Value:    [Not supported]
MultipleView.CurrentView:   [Not supported]
MultipleView.SupportedViews:    [Not supported]
ObjectModel.UnderlyingObjectModel:  [Error: calling getter for this property: hr=0xFFFFFFFF80070057 - The parameter is incorrect.]
RangeValue.IsReadOnly:  [Not supported]
RangeValue.LargeChange: [Not supported]
RangeValue.Maximum: [Not supported]
RangeValue.Minimum: [Not supported]
RangeValue.SmallChange: [Not supported]
RangeValue.Value:   [Not supported]
Selection.CanSelectMultiple:    [Not supported]
Selection.IsSelectionRequired:  [Not supported]
Selection.Selection:    [Not supported]
SelectionItem.IsSelected:   [Not supported]
SelectionItem.SelectionContainer:   [Not supported]
Scroll.HorizontallyScrollable:  [Not supported]
Scroll.HorizontalScrollPercent: [Not supported]
Scroll.HorizontalViewSize:  [Not supported]
Scroll.VerticallyScrollable:    [Not supported]
Scroll.VerticalScrollPercent:   [Not supported]
Scroll.VerticalViewSize:    [Not supported]
SpreadsheetItem.AnnotationObjects:  [Not supported]
SpreadsheetItem.AnnotationTypes:    [Not supported]
SpreadsheetItem.Formula:    [Not supported]
Style.ExtendedProperties:   [Not supported]
Style.FillColor:    [Not supported]
Style.FillPatternColor: [Not supported]
Style.FillPatternStyle: [Not supported]
Style.Shape:    [Not supported]
Style.StyleId:  [Not supported]
Style.StyleName:    [Not supported]
Value.IsReadOnly:   [Not supported]
Value.Value:    [Not supported]
Table.ColumnHeaders:    [Not supported]
Table.ItemColumnHeaderItems:    [Not supported]
Table.ItemRowHeaderItems:   [Not supported]
Table.RowHeaders:   [Not supported]
Table.RowOrColumnMajor: [Not supported]
Toggle.ToggleState: [Not supported]
Transform.CanMove:  [Not supported]
Transform.CanResize:    [Not supported]
Transform.CanRotate:    [Not supported]
Transform2.CanZoom: [Not supported]
Transform2.ZoomLevel:   [Not supported]
Transform2.ZoomMinimum: [Not supported]
Transform2.ZoomMaximum: [Not supported]
Window.CanMaximize: [Not supported]
Window.CanMinimize: [Not supported]
Window.IsModal: [Not supported]
Window.IsTopmost:   [Not supported]
Window.WindowInteractionState:  [Not supported]
Window.WindowVisualState:   [Not supported]
IsAnnotationPatternAvailable:   [Not supported]
IsDragPatternAvailable: [Not supported]
IsDockPatternAvailable: false
IsDropTargetPatternAvailable:   [Not supported]
IsExpandCollapsePatternAvailable:   false
IsGridItemPatternAvailable: false
IsGridPatternAvailable: false
IsInvokePatternAvailable:   true
IsItemContainerPatternAvailable:    false
IsLegacyIAccessiblePatternAvailable:    true
IsMultipleViewPatternAvailable: false
IsObjectModelPatternAvailable:  [Not supported]
IsRangeValuePatternAvailable:   false
IsScrollItemPatternAvailable:   false
IsScrollPatternAvailable:   false
IsSelectionItemPatternAvailable:    false
IsSelectionPatternAvailable:    false
IsSpreadsheetItemPatternAvailable:  [Not supported]
IsSpreadsheetPatternAvailable:  [Not supported]
IsStylesPatternAvailable:   [Not supported]
IsSynchronizedInputPatternAvailable:    false
IsTableItemPatternAvailable:    false
IsTablePatternAvailable:    false
IsTextChildPatternAvailable:    [Not supported]
IsTextEditPatternAvailable: [Not supported]
IsTextPatternAvailable: false
IsTextPattern2Available:    [Not supported]
IsTogglePatternAvailable:   false
IsTransformPatternAvailable:    false
IsTransform2PatternAvailable:   [Not supported]
IsValuePatternAvailable:    false
IsVirtualizedItemPatternAvailable:  false
IsWindowPatternAvailable:   false
FirstChild: "" image
LastChild:  "Link data" text
Next:   [null]
Previous:   "Cancel" button
Other Props:    Object has no additional properties
Children:   "" image
    "Link data" text
Ancestors:  "" 
    "" 
    "Link your Chrome data to this account?" 
    "Log in - company.com Ltd - Google Chrome" window
    "Desktop" 
    "Desktop" window
    [ No Parent ]

score 1 · Accepted Answer · answered Feb 13 '15 at 13:47

Your code seems just fine for me. Do you get any exception? please let me know if so.

In case you don't get any exception, the Button does not implement the InvokePattern the right way. (The way it's implemented belongs to the application under test(AUT), and to be more specific to the actually UI Automation Provider of that element.) - e.g.: the provider tells the invoke pattern to invoke right-click instead of left-click in the background which sadly can not be changed from the outside.

A work-around on that (in case you can not modify the AUT) is to use the bounding rectangle or clickablePoint propertery. Move the cursor to this location and simulate a click (by "user32.dll" for example). Take care that your window is in the foreground (which limits your script in parallel execution of several applications!)

In case you are able to change the AUT, the best is to dig into the UI Automation Provider documentation or find some tutorial on that.

thanks the user32.dll did it for me, i needed to do all of this outside of selenium in any case as i needed to interface with the real chrome! — user219396, Mar 03 '15 at 14:42
please be specific - more information regarding what? if you wanna know more about the import of "user32.dll" and it's usage you will find other SO questions like: https://stackoverflow.com/questions/8739523/directing-mouse-events-dllimportuser32-dll-click-double-click (havn't tried that very link myself, but it looks quite alright) — Haphil, Apr 07 '22 at 06:23

Chrome .Net Ui Automation - Automatic Sign on to Browser - AutomationElement Invoke appears to invoke with no result

1 Answers1

Linked