webdriver wait for ajax request in python

Question

Currently I am writing webdriver test for search which uses ajax for suggestions. Test works well if I add explicit wait after typing the search content and before pressing enter.

wd.find_element_by_xpath("//div[@class='searchbox']/input").send_keys("obama")
time.sleep(2)
wd.find_element_by_xpath("//div[@class='searchbox']/input").send_keys(Keys.RETURN)

but

wd.find_element_by_xpath("//div[@class='searchbox']/input").send_keys("obama")
wd.find_element_by_xpath("//div[@class='searchbox']/input").send_keys(Keys.RETURN)

fails. I am running tests on ec2 with 1 virtual cpu. I am suspecting, I pressed enter even before GET requests related to search are sent and if I press enter before suggestions, it fails.

Is there any better way that adding explicit waits?

score 19 · Answer 1 · answered May 20 '18 at 07:50

19

Add this method, where I ensure the API responses are back from server

def wait_for_ajax(driver):
    wait = WebDriverWait(driver, 15)
    try:
        wait.until(lambda driver: driver.execute_script('return jQuery.active') == 0)
        wait.until(lambda driver: driver.execute_script('return document.readyState') == 'complete')
    except Exception as e:
        pass

answered May 20 '18 at 07:50

Prashanth Sams

19,677
20
102
125

2

In general, bare `except` statements are not a good practice. Why do you need the `try...except...` blocks here? – reynoldsnlp Dec 28 '19 at 20:12
@reynoldsnlp it depends on your need; you can remove it if not needed. – Prashanth Sams Jul 18 '20 at 07:18

score 15 · Accepted Answer · answered Jun 05 '14 at 07:13

You indeed can add an explicit wait for the presence of an element like

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait # available since 2.4.0
from selenium.webdriver.support import expected_conditions as EC # available since 2.26.0

ff = webdriver.Firefox()
ff.get("http://somedomain/url_that_delays_loading")
ff.find_element_by_xpath("//div[@class='searchbox']/input").send_keys("obama")

try:
    element = WebDriverWait(ff, 10).until(EC.presence_of_element_located((By.ID, "keywordSuggestion")))
finally:
    ff.find_element_by_xpath("//div[@class='searchbox']/input").send_keys(Keys.RETURN)
    ff.quit()

See: http://docs.seleniumhq.org/docs/04_webdriver_advanced.jsp#explicit-and-implicit-waits

score 0 · Answer 3 · answered May 12 '23 at 08:46

You can use selenium-wire instead which is a wrapper of selenium. It will setup a proxy with port 8087 in client for webdriver and listen all the requests. The code is like

from seleniumwire import webdriver

chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--proxy-server=127.0.0.1:8087')
driver = webdriver.Remote(
    command_executor="http://127.0.0.1:4444/wd/hub",
    options=chrome_options,
    seleniumwire_options={
        'auto_config': False,
        'port': 8087,
        'addr': '127.0.0.1',
    }
)

score -4 · Answer 4 · answered Sep 27 '14 at 11:29

And what about:

    driver.implicitly_wait(10)

for your example:

    wd.implicitly_wait(10)

In this case every time you are going to click or find element driver will try to do this action every 0.5 second during 10 seconds. In this case you don't need to add wait every time. Note: But it is only about element on screen. It will not wait until some JS actions ends.

webdriver wait for ajax request in python

4 Answers4

Linked