How can I find an element by screen location with Python Selenium?

Question

I've been searching online but haven't found any answers yet on how to select an element by screen position with selenium. I've found ways to get the position of an element once you've selected it but not a way to select it from its screen position.

My end goal is to make a web macro recorder to do repetitive tasks, so my idea was to use pyautogui to get the screen location of the users click, and then be able to convert that position with selenium to a CSS selector (the part I haven't figured out yet) which would help my script be a lot more robust.

In the worst of cases if what I am asking is impossible, I think I could parse the html with beautifulsoup and then compare the pyautogui click with every element's position with selenium but this would make every thing so much slower so I'd like to avoid that option at all costs.

Any help or interesting feedback from you all out there would be greatly appreciated!

Answer 1

With JavaScript you'll be able to know the topmost Element at the specified coordinates.

var element = document.elementFromPoint(x, y);

See elementFromPoint.

Then you might try to Build the querySelector string value of any given node in the DOM.

---

And this is tutorial on how to execute JavaScript with Selenium/Python:

https://www.softwaretestingmaterial.com/javascript-executor-in-selenium-python/