1

I'm trying to use web scraping to get the parking price at this link, https://application.parkbytext.com/accountus/prepay.htm?locationCode=1127. It's $2 for the day which is what I'm trying to get. I'm using python+selenium, but just couldn't get the parking price. Below is the code I'm using, but sometimes I hit the target but most time I get the error

selenium.common.exceptions.NoSuchElementException: Message: Unable to locate element: {"method":"class name","selector":"gwt-RadioButton"}.

Can anyone help? thanks in advance

def downtownparking(driver):
driver.get("https://application.parkbytext.com/accountus/prepay.htm?locationCode=1127")
try:
    ### driver.wait = WebDriverWait(driver, 16)
    ### driver.implicitly_wait(20)
    cr = driver.find_element_by_class_name("gwt-RadioButton")
    dayprice = cr.find_element_by_tag_name("label")
    print (dayprice.text)
Nazmul Hasan
  • 10,130
  • 7
  • 50
  • 73
teapot
  • 77
  • 5

1 Answers1

1

The page loading takes time. At the moment webdriver tries to find an element, it is not yet present in the DOM tree. Add an Explicit Wait:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

cr = WebDriverWait(driver, 10).until(
    EC.presence_of_element_located((By.CLASS_NAME, "gwt-RadioButton"))
)

As a side, note, I would use the input's name instead:

cr = WebDriverWait(driver, 10).until(
    EC.presence_of_element_located((By.XPATH, "//input[@name='periodType']/following-sibling::label"))
)
print(cr.text)  # prints "Day - $2.00"
alecxe
  • 462,703
  • 120
  • 1,088
  • 1,195
  • Thanks for the great help. It works. Actually I was wondering the wait was the reason. So I tried driver.wait = WebDriverWait(driver, 16) or driver.implicitly_wait(20), but didn't work. Do you know why the explicit or implicit wait doesn't work in this case? – teapot Jan 28 '16 at 19:14
  • @teapot I think this is a pretty good overview: http://stackoverflow.com/questions/10404160/when-to-use-explicit-wait-vs-implicit-wait-in-selenium-webdriver. Also, please see http://stackoverflow.com/help/someone-answers. – alecxe Jan 28 '16 at 19:16
  • Sure I'll accept the answer. I just need to wait at least 10 minutes to click the button. You are awesome! Thanks for the overview link, I'll take a look. – teapot Jan 28 '16 at 19:18
  • oops, the revision didn't work. The first answer works though. – teapot Jan 28 '16 at 19:22