Selenium python error : element is not attached to the page document

Question

I need to scrape data from link by choosing start_date and end_date. Once the dates are chosen, there are several links in a table. Once a link is clicked, a popup box comes up and I scrape the data from that pop-up box.

My code is able to scrape the data from the pop-up box, but after doing this for the first link, it runs into an error :

selenium.common.exceptions.StaleElementReferenceException: Message: stale element reference: element is not attached to the page document
  (Session info: chrome=84.0.4147.125)

The error points to line 41

element.click()

This is my code so far :

from time import sleep
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import csv
import os
import sys
start_date, end_date = '1960-01-01', '1970-01-01'

driver = webdriver.Chrome()
driver.get('http://rni.nic.in/registerdtitle_search/registeredtitle_ser.aspx')
driver.maximize_window()

date_button = driver.find_element_by_xpath('//*[@id="main_TabContainer2_tbpnljobdetails_Label41"]')
date_button.click()

start_dater = driver.find_element_by_xpath('//*[@id="main_TabContainer2_tbpnljobdetails_txt_fromdate"]').send_keys(start_date)
end_dater = driver.find_element_by_xpath('//*[@id="main_TabContainer2_tbpnljobdetails_txt_todate"]').send_keys(end_date)

search_button = driver.find_element_by_xpath('//*[@id="main_TabContainer2_tbpnljobdetails_btn_search3"]')
search_button.click()

table = driver.find_element_by_xpath('/html/body/form/div[4]/div[2]/div[2]/div/div[2]/div[3]/div/div/div[3]/div/table')

titles = ['TITLE','REGN NO','LANGUAGE','PERIODICITY', 'STATE', 'PUBLICATION DISTRICT', 'PRICE',
'PUBLISHER NAME', 'PUBLISHER ADDRESS', 'PRINTER NAME', 'PRINTER ADDRESS', 'EDITOR NAME',
'EDITOR ADDRESS', 'PRINTING PRESS NAME', 'PRINTING PRESS ADDRESS', 'OWNER NAME',
'PLACE OF PUBLICATION', 'PUBLISHER EMAIL ID', 'PUBLISHER MOBILE NO', 'OWNER EMAIL ID',
'OWNER MOBILE NO','OWNER LANDLINE NO']

sleep(4)
driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")

filename = start_date + ' to ' + end_date + '.csv'
with open(filename, 'w') as f:
    writer = csv.writer(f)
    writer.writerow(titles)
    for element in driver.find_elements_by_class_name('popup_registrationd'):
        dic = {key:'' for key in titles}
        element.click()
        popup = driver.find_element_by_xpath('//*[@id="div1"]')
        for content in popup.find_elements_by_css_selector('div'):
            contents = content.find_elements_by_css_selector('span')
            data = [c.text for c in contents]
            for key,value in zip(data[0::2], data[1::2]):
                dic[key] = value
        writer.writerow(list(dic.values()))
        print(list(dic.values()))
        closer = driver.find_element_by_xpath('//*[@id="main_btnClose"]')
        closer.click()

Have you researched what a stale element is? Once you understand how it happens, you will be able to fix the issue. This question has been asked and answered many times before. — JeffC, Aug 14 '20 at 18:03
I'll look into it @JeffC, I just don't understand why the state element is not available as it's still in the same page. — Suraj, Aug 14 '20 at 18:04
I solved it. Instead of finding this : `driver.find_elements_by_class_name('popup_registrationd')` outside the for loop. I modified the code so I could find it in each iteration. I also used a `total` and `current_count` variable to index the element I require from the table. — Suraj, Aug 15 '20 at 06:10

Selenium python error : element is not attached to the page document

0 Answers0