UI Automation - ElementFromHandle doesn't find element

Question

I am trying to use UIAutomation to access/control Chrome browser (using that method based on what other people have used - eg to get the current URL).

For the purposes ok the exercise, I'm trying to replicate this question - Retrieve current URL from C# windows forms application - in Delphi. I've imported the TLB ok. However, my call to ElementFromHandle never locates an element.

The signature of the ElementFromHandle method is:

function ElementFromHandle(hwnd: Pointer; out element: IUIAutomationElement): HResult; stdcall;

My test is simply:

procedure TForm3.Button1Click(Sender: TObject);
var
  UIAuto: IUIAutomation;
  element: IUIAutomationElement;
  value: WideString;
  h: PInteger;
begin
  new(h);
  h^ := $1094E;
  SetForegroundWindow(h^);
  ShowWindow(h^, SW_SHOW);
  UIAuto := CoCUIAutomation.Create;
  UIAuto.ElementFromHandle(h, element);
  if Assigned(element) then
  begin
    element.Get_CurrentName(value);
    showmessage('found -' + value);
  end
  else
    showMessage('not found');
end;

Calls to SetForegroundWindow and ShowWindow are just there in case it needed focus (but I doubted that it would make a difference and doesn't). I can confirm that the Handle I'm passing in ($1094E) is "correct" in so much as Spy++ shows that value for the Chrome Tab I'm trying to access. The active tab in Chrome always reports that Handle.

Is my implementation correct above? Is there more to using UIAutomation than what I have implemented above? I have never explored it before.

Thanks

EDIT

I have found if I use ElementFromPoint and pass in a (hardcoded) value of where I know my Tab sits in terms of X,Y - it does work. ie:

  UIAuto := CoCUIAutomation.Create;
  p.x := 2916;
  p.y := 129;
  UIAuto.ElementFromPoint(p, element);
  if Assigned(element) then

The above snippet if placed in the above OnClick event does return an element instance and the one I'm expecting too (which is a bonus). So maybe I'm passing in an incorrect value for Hwnd in ElementFromHandle? ie, I'm using the "top" level handle of Chrome as found my MS Spy++:

This sits directly under (Desktop) in Spy++.

Answer 1

Your mistake is in the way that you pass the window handle to ElementFromHandle. You are meant to pass an HWND. Instead you pass the address of an HWND.

The function should really be:

function ElementFromHandle(hwnd: HWND; 
    out element: IUIAutomationElement): HResult; stdcall;

You should remove the call to New and instead do:

var
  window: HWND;
....
window := HWND($1094E);

Then call the function like this:

if Succeeded(UIAuto.ElementFromHandle(window, element)) then
  ....

Perhaps your biggest fundamental problem is the complete absence of error checking. I think you need to adjust your mindset to realise that these API calls will not raise exceptions. They report failure through their return value. You must check every single API call for failure.

One common way to do that is to convert HRESULT values to exceptions in case of failure with calls to OleCheck. For example:

var
  UIAuto: IUIAutomation;
  element: IUIAutomationElement;
  value: WideString;
  window: HWND;
....
window := HWND($1094E);
SetForegroundWindow(window);
ShowWindow(window, SW_SHOW);
UIAuto := CoCUIAutomation.Create;
OleCheck(UIAuto.ElementFromHandle(window, element));
OleCheck(element.Get_CurrentName(value));
ShowMessage('found -' + value);