1

How to reduce time of explicit wait of a page with full of AJAX statements?
wait=WebDriverWait(driver,timeout=77) gives an exact results of located elements that I need, but consumed time is huge. When I set timeout time to lower than 70 the webdriver can not locate elements gives unable to locate elements.

Prophet
  • 32,350
  • 22
  • 54
  • 79
xlmaster
  • 659
  • 7
  • 23

1 Answers1

1

There are some web pages having some JavaScripts there, making those pages loading long time.
Selenium has 3 possible settings for the Page loading Strategy. The default strategy is normal. With this setting Selenium will wait for complete document readiness state of the loaded page. To reduce the loading time with Selenium we normally use eager strategy setting. This will wait for interactive document readiness state of the loaded page.
Here some simple working example of how to set page strategy to eager with Selenium on Python:

from selenium import webdriver
from selenium.webdriver import DesiredCapabilities
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
options = Options()
options.add_argument("start-maximized")

caps = DesiredCapabilities().CHROME
caps["pageLoadStrategy"] = "eager"

webdriver_service = Service('C:\webdrivers\chromedriver.exe')
driver = webdriver.Chrome(service=webdriver_service, options=options, desired_capabilities=caps)
Prophet
  • 32,350
  • 22
  • 54
  • 79
  • Concise, clear explanation of concept and easy to use start code snippets. Thanks @Prophet – xlmaster Nov 04 '22 at 11:37
  • strange, it immediately opened first page and immediately clicked first link , but in second directed page it gave an error. I take there img link it couldn't locate element. Maybe I do not use there wait explicitly i just try to locate elelemnt with `driver.find_..` let me change that to `wait.until(EC.visibi..` then try again – xlmaster Nov 04 '22 at 11:49
  • `timeoutException` I was wrong not the img tag locating line, but `wait.until(EC.visibility_of_all_elements_located((By.XPATH,'//span[contains(@class,"isbn")]/parent::node()/following-sibling::div/span')))` line befor it was `driver.find_elements(By.XPATH,'//span[contains(@class,"isbn")]/parent::node()/following-sibling::div/span')`. Can it be due to Xpath functions like contains(), text() – xlmaster Nov 04 '22 at 11:58
  • changed `visibililty of all elements` to `presence of all elements`strangely worked. But in all above places I must be used visibility because either way it was giving an error...strangeness all along – xlmaster Nov 04 '22 at 12:03
  • 1
    Not strange. There are a lot of elements on the web pages that are not defined as visible. probably you could use some other elements near those you are locating, that would be visible. You definitely should use `visibililty of all elements` in such cases since `presence of all elements` can catch element that are still not fully rendered but already existing on the page. – Prophet Nov 04 '22 at 12:13
  • 1
    So, I'd suggest using other elements to wait for their visibility or maybe using `presence of all elements` with some short sleep after that and only after that I'd collect the list of those elements. This looks ugly but will be more stable than using just `presence of all elements` – Prophet Nov 04 '22 at 12:13
  • 1
    now it is clear, the different results between these two features – xlmaster Nov 04 '22 at 12:18
  • I did as your suggestion: Today I implemented adding time.sleep() after explicitly wait(). After going through pages, starting from second page it was not loading even though exlicitly waiting, so I added sleep(). Works properly!..Great to comeback, and reuse your advices) – xlmaster Nov 06 '22 at 14:29
  • 1
    Very nice to know that, thanks! – Prophet Nov 06 '22 at 15:00