1

Problem similar to the one described here.

I want to get the mouse from point a to point b along a curve that I define. So, I need a and b's location coordinates. Point a is the mouse's current location, and point b is an element that I can locate. I can find the element's position using the following: 

element = driver.find_element_by_class_name("insertnamehere")
x, y = element.location["x"], element.location["y"]

Unfortunately, I can't use similar Selenium methods to find the mouse's current location (as far as I know). So I have to find the mouse like this:

import pyautogui
x_mouse, y_mouse = pyautogui.position()

Pyautogui gives screen location, which--as the above linked question described--is not what element.location gives. But the linked solution doesn't work for me on the site that I'm interested in. That is, I don't get matching coordinates. The site is this one.

I suspect it's because Selenium's locator doesn't count the map element as part of the web page, and returns coordinates relative to the listings (the part that you can scroll). So, here are some ideas I've tried:

  • Convert the screen location into Selenium location, or vice versa.

This is difficult because I think the formula (screen to Selenium) will depend on my monitor, so won't work generally.

  • Put the mouse on a web element so that point a can be located using Selenium's .location.

Might be the best solution, but I would have to move to the element using ActionChains(driver).move_to_element(), which defeats the purpose of defining an a -> b mouse path.

  • Either find the mouse location using Selenium's location method or find the element in terms of screen coordinates.

I've tried pyautogui's locateOnScreen(). Seems promising, and I will probably just do it this way.

Anshu Chen
  • 423
  • 2
  • 9
  • 23

1 Answers1

4

If you are trying to get the absolute x and y values (x and y values relative to screen) then use the below logic.

# maximize the browser window so that you can get the absolute x and y based on the window inner and outer heights
driver.maximize_window()
# navigate to OP
driver.get("https://stackoverflow.com/questions/56688961/locating-elements-screen-position-in-selenium-using-python")
# get ask question element
askQuestion = driver.find_element_by_link_text("Ask Question")
# get the browser panel height
panel_height = driver.execute_script('return window.outerHeight - window.innerHeight;')
# get absolute x value
abs_x = askQuestion.location['x']

# get y value (this is relative y value - meaning y value with in browser)
y = askQuestion.location['y']
abs_y = y + panel_height
# print the absolute x and y values
print ("Absolute x : " + abs_x)
print ("Absolute y : " + abs_y)
supputuri
  • 13,644
  • 2
  • 21
  • 39
  • This didn't work when I tried it on zillow.com/homes. I used the code you provided to locate this element: ```pgnext = driver.find_element_by_class_name("zsg-pagination-next")```, which gave me [965, 459] in screen coordinates. But when I located the element in screen coordinates using pyautogui's locateCenterOnScreen, I got [1620, 675]. Not sure what's causing the discrepancy. – Anshu Chen Jun 21 '19 at 17:57
  • What's puzzling is that the code clearly works on most sites, so there's probably something in Zillow that's screwing it up... – Anshu Chen Jun 21 '19 at 18:00