1

I'm trying to use Selenium & Python to scrape a website (http://epl.squawka.com/english-premier-league/06-03-2017/west-ham-vs-chelsea/matches). I am using the webdriver to click a heading and then wait for the new information to load before clicking on an object before scraping the resulting data (which loads from the clicking). My problem is that I keep on getting an 'Unable to locate element error.

I've taken a screenshot at this point and can physically see the element and I've also printed the entire source code and can see that the element is there.

driver.find_element_by_id("mc-stat-shot").click()
time.sleep(3)
driver.save_screenshot('test.png')
try:
    element = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.ID,"svg")))
finally:
    driver.find_element_by_xpath("//g[3]/circle").click()
time.sleep(1)
goalSource = driver.page_source
goalBsObj = BeautifulSoup(goalSource, "html.parser")
#print(goalBsObj)
print(goalBsObj.find(id="tt-mins").get_text())
print(goalBsObj.find(id="tt-event").get_text())
print(goalBsObj.find(id="tt-playerA").get_text())

and the result is an error: "selenium.common.exceptions.NoSuchElementException: Message: Unable to locate element: //g[3]/circle"

Wayde Herman
  • 71
  • 5
  • Use explicit waits for the page element to be ready. See: https://stackoverflow.com/questions/10404160/when-to-use-explicit-wait-vs-implicit-wait-in-selenium-webdriver – N Shumway Jun 16 '17 at 10:09
  • What element are you trying to locate with the xpath: //g[3]/circle ?? – Wonka Jun 16 '17 at 10:40
  • The specific shots. – Wayde Herman Jun 16 '17 at 10:44
  • @NShumway it doesn't fix it. I just edited the question with that included. Thanks though! – Wayde Herman Jun 16 '17 at 10:59
  • @Wayde Herman how do you check that xpath works? I test with firebug+firepath and there is no element matching you Xpath! No element like anyway... If you say me what are you trying to locate I will help you with the xpath – Wonka Jun 16 '17 at 11:07
  • @Wonka i really hope its because I made a stupid mistake. This is the full xpath: //*[@id="mc-pitch-view"]/div[2]/div[1]/svg/g[2] (It only exists after you click 'shots') – Wayde Herman Jun 16 '17 at 11:21
  • 1
    @WaydeHerman The arrays are 0 indexed. Your xpath looks like you need to subtract 1 from each index – chris-crush-code Jun 16 '17 at 15:17

0 Answers0