0

I am trying to retrieve all the links to the posts of on instagram account, however it breaks and I am currently struggling to write the proper WebDriverWait so I could make it work.

My few attempts have taken me as close as that:

    links = WebDriverWait(posts, 10).until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, ".a")))
    links = [elem.get_attribute('href') for elem in links]

But it throws NameError: name 'By' is not defined

The code that I am trying to run before any adjustments were made:

    browser= webdriver.Chrome('/Users/marialavrovskaa/desktop/chromedriver')
browser.get(f"{instagram_page}")
while scheight < 9.9:
    browser.execute_script("window.scrollTo(0, document.body.scrollHeight/%s);" % scheight)
    scheight += .01
    posts += browser.find_elements_by_xpath(f"//div[@class='{xpath_to_links}']")

Here the code breaks:

links = elem.find_element_by_css_selector('a').get_attribute('href') for elem in posts]
    print(links)

I've built my understanding around that error, yet I struggle to make it work with By.CSS_SELECTOR settings. Any help will be greatly appreciated.

Community
  • 1
  • 1
Karina
  • 29
  • 6
  • "But it throws NameError: name 'By' is not defined" Have you tried: from selenium.webdriver.common.by import By – Jordan Mar 30 '20 at 11:47
  • Yes, still the same. – Karina Mar 30 '20 at 11:47
  • Well then what error does it throw? It can't be the same error – Jordan Mar 30 '20 at 11:58
  • links = [WebDriverWait(elem, 10).until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, ".a"))) for elem in posts] this throws Message: stale element reference: element is not attached to the page document (Session info: chrome=80.0.3987.149) – Karina Mar 30 '20 at 12:47
  • But my question is mainly that I am not sure how should I rewrite and iterate throw posts By CSS_SELECTOR so it will still collect the links – Karina Mar 30 '20 at 12:48

0 Answers0