I'm trying to scrape Glassdoor for job postings using Selenium. My script works for roughly 20~ postings until I get the following error:
ElementClickInterceptedException: Message: element click intercepted: Element <li class="jl...</li> is not clickable at point (219, 699). Other element would receive the click: <div class="gdGrid noPad">...</div>
The element that would receive the click is a grid surrounding the postings. I'm pretty sure it's not obscuring the postings. I'm not sure why it randomly can't click the next job posting. I've tried changing slp.time but nothing works. Below is the code I'm using to click on each posting.
#Going through each job in this page
job_buttons = driver.find_elements_by_class_name("jl") #jl for Job Listing. These are the buttons we're going to click.
for job_button in job_buttons:
print("Progress: {}".format("" + str(len(jobs)) + "/" + str(num_jobs)))
if len(jobs) >= num_jobs:
break
job_button.click() #You might
time.sleep(5)
collected_successfully = False
while not collected_successfully:
try:
company_name = driver.find_element_by_xpath('.//div[@class="employerName"]').text
location = driver.find_element_by_xpath('.//div[@class="location"]').text
job_title = driver.find_element_by_xpath('.//div[contains(@class, "title")]').text
job_description = driver.find_element_by_xpath('.//div[@class="jobDescriptionContent desc"]').text
collected_successfully = True
except:
time.sleep(5)
This is where I'm getting the error:
---> 60 job_button.click() #You might
61 time.sleep(5)
62 collected_successfully = False
Any solutions? I'm using Chrome