I am scraping 'href' tags from a website while inputting values from a csv file. The issue that I am facing is that the webpage sometimes doesn't get loaded fully and is returning values from the previously loaded webpage. For the same, I want to ensure that the webpage is loaded fully. I am using selenium in python and have used the following query:
links = driver.find_elements_by_xpath("//*[@class='search-result__image-wrapper']/a")
WebDriverWait(driver, timeout).until(links[0])
Basically, the links variable extracts the elements that contain the profile URLs if it exists. There might be cases where "links" is null. The above code gives me the following error:
"WebElement object is not callable"
When I tried using,
links = driver.find_elements_by_xpath("//*[@class='search-result__image-wrapper']/a")
WebDriverWait(driver, timeout).until(links)
The error is:
"TypeError: List object is not callable in python"
Can you please help me with this? I am new to python and web scraping. Thanks!
