How do I make Selenium Driver in Python wait until .get() page loads fully to get entire page source?

Question

My Selenium Driver is suddenly receiving None when requesting from a page. I want to store urlx inside dictionary data but my driver.get(x) is returning None.

self.data= {
"google.com": page_source,
"stackoverflow.com/....": page_source,
etc...
}

I tried using

self.data[link] = self.driver.get(link)
            
  try:
       self.data[link] = WebDriverWait(self.driver, 10).until(EC.presence_of_element_located(By.NAME, "?"))
  finally:
       continue

but it doesnt seem to stop and wait, also I don't want to wait for a specific element, I want for the entire page to finish loading.

I've also tried to use driver.implicity_wait(10) but it doesnt seem to wait at all.

How do I do this?

score 0 · Answer 1 · answered Nov 26 '22 at 14:00

Reference this answer to the above question.

driver.get("https://stackoverflow.com")
pageSource = driver.page_source
fileToWrite = open("page_source.html", "w")
fileToWrite.write(pageSource)
fileToWrite.close()
fileToRead = open("page_source.html", "r")
print(fileToRead.read())
fileToRead.close()
driver.quit()

How do I make Selenium Driver in Python wait until .get() page loads fully to get entire page source?

1 Answers1