I am trying to scrape images from website and using selenium but getting an error in the code

Question

My code

for idx, row in tqdm(df.iterrows(), total=df.shape[0]):
    imgs = []
    try:
        driver.get(row['listing_url'])
        sleep(10)
        script = "document.getElementsByClassName('accordion-toggle')[0].click();"
        driver.execute_script(script)
        sleep(10)
        scroll_down(driver)
        imgs = driver.find_elements(By.XPATH, '//img[@class="i1o0kbi8 i1mla2as i1cqnm0r dir dir-ltr"]')
        imgs = list(set([img.get_attribute('src') for img in imgs]))
    finally:
        os.makedirs('images', exist_ok=True)
        with open(f'images/{row["id"]}_{row["host_id"]}.json', 'w') as f:
            json.dump(imgs, f)

Error

JavascriptException: Message: javascript error: Cannot read properties of undefined (reading 'click')

What is wrong with my code and what is the correct approach

`document.getElementsByClassName('accordion-toggle')` isn't finding anything, so there's nothing in the `[0]` element to click on. — Barmar, Jun 12 '23 at 20:24

score 1 · Answer 1 · answered Jun 12 '23 at 20:23

This error message...

JavascriptException: Message: javascript error: Cannot read properties of undefined (reading 'click')

...implies that click() method can't be invoked on undefined element.

This usecase

Possibly the following part of the script:

document.getElementsByClassName('accordion-toggle')[0]

doesn't identifies any element, so remains undefined and click() method can't be invoked on it.

Solution

Try to identify the desired element uniquely within the HTML DOM using a suitable locator strategy and execute your code.

I am trying to scrape images from website and using selenium but getting an error in the code

1 Answers1

This usecase

Solution