0

I am trying to put together a web scraper to get locations by zip code entered by the user. As of right now I am able to navagate to the website but I am not able to click on the drop down button that allows you to enter in a zip code. Here is what I have so far

from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from bs4 import BeautifulSoup
import time
import pandas as pd
from selenium.webdriver.common.by import By

zipcode = input("What zip code would you like to search? ")
out_table = 'Oreilly_autp_parts_addresses_{}.csv'.format(zipcode)

#Using Selenium to navigate to website, search zipcode and get html data
driver = webdriver.Chrome() #requires geckodriver.exe
driver.get('https://www.oreillyauto.com/')
time.sleep(2)
driver.maximize_window()
el = driver.find_element_by_class_name("site-store")
time.sleep(2)
driver.execute_script("arguments[0].setAttribute('class','site-store site-nav_item dispatcher-trigger--active')", el)

It seems to be clicking on the correct element but the drop down that is supposed to show up isn't there. HTML Code Block

Any help is much appreciated!

jnord
  • 55
  • 6
  • I think that site is detecting webdriver and not allowing certain behavior. When I launch it without webdriver I can click the store locator, but if I launch the browser with chromedriver I cannot manually click it. – Ywapom Sep 06 '18 at 21:08
  • Yeah I've noticed that too. Do you think that means its not possible to automate this site? – jnord Sep 06 '18 at 21:31
  • There are posts on how to get around the detection. Just depends how much time you are willing to spend. – Ywapom Sep 07 '18 at 15:53
  • @Ywapom could you possibly point me in the right direction? – jnord Sep 07 '18 at 19:59
  • https://stackoverflow.com/questions/33225947/can-a-website-detect-when-you-are-using-selenium-with-chromedriver – Ywapom Sep 07 '18 at 21:43

0 Answers0