Get notifications of movement of control with UI Automation

Question

Is there a way to track the screen coordinates of a control using the UI Automation API in .NET?

I can find out where it is once, but once I've got hold of the control I need to be notified if its position on the screen changes. Is this possible?

Answer 1

I'd recommend you point the AccEvent SDK tool to the UI you're interested in, to see what UIA events are generated when the control moves. Different UI frameworks might raise different events, and sometimes it can be impossible to predict exactly what events you'll have a chance to respond to.

For example, I just ran AccEvent and told it to tell me about every event that’s raised beneath the Control Panel UI when I resized the Control Panel. The screenshot below shows that a bucket-load of UIA PropertyChanged events are raised for UIA elements’ BoundingRectangle properties, and also a few StructureChanged events too.

Depending on your situation, you may need to listen for events raised both on the control itself and also on the window that contains it. For example, say I want to know when the Browse button in the Run dlg moves. If I drag the window around, I get no events raised by the Browse button, but I do get lots of events saying that the Run dlg window itself is moving, (as shown in the screenshot below). If the window is moving, then I know the Browse button's position on the screen is also changing.

So once you've used AccEvent to learn about which events are raised, you can write your UIA client code to handle those events and achieve your goals. AccEvent is a UIA client app, so any event shown in AccEvent is also something that your own UIA client code can react to.

I put a few notes on where UIA event handlers need to be handled with care at UI Automation events stop being received after a while monitoring an application and then restart after some time.

Thanks,

Guy