How can I use WebDriverWait from Selenium properly through Python?

Question

I just thought I'd add an edit now that this has been resolved. Replacing those two time.sleep() took my program from 180 seconds down to 30. WebDriverWait creates a substantial improvement in runtime.

I am just trying to determine if I'm setting up WebDriverWait correctly. This is my working script, I use time.sleep()

for x,sequence in enumerate(table.find_elements_by_xpath('//*[@id="gwzSngrOrderResultPanelRoot"]/table/tbody/tr/td[9]'),1):
        driver.find_element_by_xpath(f'//*[@id="gwzSngrOrderResultPanelRoot"]/table/tbody/tr[{x}]/td[9]/span[2]').click()
        time.sleep(5)
        element = WebDriverWait(driver, 5).until(EC.presence_of_element_located((By.XPATH, '//*[@id="gwzViewResultsModalDialog"]/div/div/div[3]/button')))
        seq_info=driver.find_element_by_xpath('//*[@id="gwzViewResultsModalDialog"]/div/div/div[2]/div')
        seq_list.append([seq_info.text])
        driver.find_element_by_xpath('//*[@id="gwzViewResultsModalDialog"]/div/div/div[3]/button').click()
        time.sleep(5)

In short, it goes through a table, clicks a button which opens a pop up, extracts text from pop up, and closes said pop up. I have to wait for the pop up to open, and to fully close. I'm currently using time.sleep(), but I'm trying to switch to WebDriverWait. This is how I've implemented it.

for x,sequence in enumerate(table.find_elements_by_xpath('//*[@id="gwzSngrOrderResultPanelRoot"]/table/tbody/tr/td[9]'),1):
        driver.find_element_by_xpath(f'//*[@id="gwzSngrOrderResultPanelRoot"]/table/tbody/tr[{x}]/td[9]/span[2]').click()
        element = WebDriverWait(driver, 5).until(EC.presence_of_element_located((By.XPATH, '//*[@id="gwzViewResultsModalDialog"]/div/div/div[3]/button')))
        seq_info=driver.find_element_by_xpath('//*[@id="gwzViewResultsModalDialog"]/div/div/div[2]/div')
        seq_list.append([seq_info.text])
        driver.find_element_by_xpath('//*[@id="gwzViewResultsModalDialog"]/div/div/div[3]/button').click()
        time.sleep(5)

However, the above does not work. And I receive this error:

driver.find_element_by_xpath('//*[@id="gwzViewResultsModalDialog"]/div/div/div[3]/button').click()
selenium.common.exceptions.ElementNotInteractableException: Message: element not interactable

This is gone if I return to time.sleep(), thus making me think I must have setup my WebDriverWait incorrectly. The waiting is for the opening and closing of the browser, so we have to wait till the button appears, thus I put the xpath of the button itself in the WebDriverWait. Is this the proper setup?

Edit: Thank you @DebanjanB for the answer. However, I have come across another issue in trying to remove my time.sleep(), this is what I currently have.

for x,sequence in enumerate(table.find_elements_by_xpath('//*[@id="gwzSngrOrderResultPanelRoot"]/table/tbody/tr/td[9]'),1):
        driver.find_element_by_xpath(f'//*[@id="gwzSngrOrderResultPanelRoot"]/table/tbody/tr[{x}]/td[9]/span[2]').click()
        time.sleep(5)
        #WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, '//*[@id="gwzViewResultsModalDialog"]/div/div/div[2]/div')))
        seq_info=driver.find_element_by_xpath('//*[@id="gwzViewResultsModalDialog"]/div/div/div[2]/div')
        seq_list.append([seq_info.text])
        WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, '//*[@id="gwzViewResultsModalDialog"]/div/div/div[3]/button'))).click()

The time.sleep() above is waiting for the pop up to open (from the previous click), before it extracts the data. If I remove the time.sleep with the below #WebDriver, it still moves forward, but for some reason seq_info.text is now blank (it no longer finds the text). I don't quite understand why this is happening. This is not a click or button, I just wanted to check that the pop up is open before I extract the info from it.

https://imgur.com/a/fKsiw6a I'm clicking on the clipboard in seq_file, that opens up a pop up on the same page with the info to extract — samman, Jul 06 '20 at 14:44

score 0 · Accepted Answer · edited Oct 31 '22 at 23:06

0

Short answer, no, though syntactically correct, but you aren't using WebDriverWait optimally.

Along with WebDriverWait you are also using time.sleep().

time.sleep(secs)

time.sleep(secs) suspends the execution of the current thread for the given number of seconds. The argument may be a floating point number to indicate a more precise sleep time. The actual suspension time may be less than that requested because any caught signal will terminate the sleep() following execution of that signal’s catching routine. Also, the suspension time may be longer than requested by an arbitrary amount because of the scheduling of other activity in the system.

You can find a detailed discussion in How to sleep webdriver in python for milliseconds

Moreover,

In the for loop as you intend to iterate instead of /tr[{x}] you need //tr[{x}]
To collect the desired text you you need to use visibility_of_element_located().
<button> are interactive in nature, so instead of presence_of_element_located() you need to use element_to_be_clickable() just when you need to interact with them.

A probable solution:

for x,sequence in enumerate(table.find_elements_by_xpath('//*[@id="gwzSngrOrderResultPanelRoot"]/table/tbody/tr/td[9]'),1):
      WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, '//*[@id="gwzSngrOrderResultPanelRoot"]/table/tbody//tr[{x}]/td[9]/span[2]'))).click()
      seq_list.append(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//*[@id='gwzViewResultsModalDialog']/div/div/div[2]/div"))).text)
      WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, '//*[@id="gwzViewResultsModalDialog"]/div/div/div[3]/button'))).click()

edited Oct 31 '22 at 23:06

Peter Mortensen

30,738
21
105
131

answered Jul 06 '20 at 14:34

undetected Selenium

183,867
41
278
352

I was only using time.sleep() because I wanted to change things 1 step at a time (I know time.sleep() works, but I don't know how to use WebDriverWait, thus if the program failed, I knew exactly where/why it failed). I agree WebDriverWait is far better than time.sleep(), however it is not working in my above situation. Is it because I am also using time.sleep() as well? And if I am not using it optimally, how may I optimize its usage? – samman Jul 06 '20 at 14:43
1

@samman Checkout the updated answer and let me know the status. – undetected Selenium Jul 06 '20 at 15:03
Oh, so can WebDriverWait also click the object? I.E. Replace the .click() option? Also, why //tr instead of /tr? /tr was working for me? – samman Jul 06 '20 at 15:08
So element_to_be_clickable worked (although, surprisingly, it did not effect the run time of my program. Also, the first WebDriverWait is unneeded, since the table is already found, thus theres no need for a timer to click the button. The time.sleep() was for the pop up to load, so I've added: ```WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, '//*[@id="gwzViewResultsModalDialog"]/div/div/div[2]/div')))``` (the xpath to the pop up) right before seq_info, however when I do this, seq_info now becomes blank for some reason (i.e. seq_info txt gives an empty string) – samman Jul 06 '20 at 15:53
1

@samman 1) `element_to_be_clickable()` is effective, efficient and proven, there shouldn't be any doubts. 2) `click()` should be always invoked through _WebDriverWait_, no matter it's a `` or a ` – undetected Selenium Jul 06 '20 at 16:54
I think it's clearer.easier to see what I'm trying to say if I just edit my question. I edited my question (the bottom half) to address what I was trying to say. – samman Jul 06 '20 at 17:05
1

@samman Checkout the updated answer and let me know the status. – undetected Selenium Jul 06 '20 at 17:12
That worked perfectly (although, the first click is an f string, that's the only thing missing). Thank you so much. Just out of curiosity, is it better to just use WebDriverWait for all clicks in general? I.E. Should that just be the default? Seems like it would save quite a bit of troubleshooting since you'll no longer get wait errors – samman Jul 06 '20 at 17:44
@samman _WebDriverWait_ always for `click()` and the _timespan_ should be mentioned in the _Test Plan_. – undetected Selenium Jul 06 '20 at 17:47

How can I use WebDriverWait from Selenium properly through Python?

1 Answers1

time.sleep(secs)

Linked