11

I just wonder, how to have the browser wait before clicking on a link? My goal is that I'm scraping from a dynamic webpage, the content is dynamic but I manage to get the form id. The only problem is that the submit button is only displayed after 2-3 seconds. However, my Firefox driver start clicking on the link immediately when the page is loaded (not the dynamic part).

Is there any way that I can make my browser wait 2-3 seconds until the submit button appears? I tried to use time.sleep() but it pauses everything, the submit button doesn't appear during time.sleep but appears after 2-3 seconds when time.sleep ends.

MrWhite
  • 43,179
  • 8
  • 60
  • 84
Kiddo
  • 1,910
  • 8
  • 30
  • 54

2 Answers2

18

You can set wait like following :

Explicit wait : 

    element = WebDriverWait(driver, 20).until(
    EC.presence_of_element_located((By.ID, "myElement"))

Implicit wait :

 driver.implicitly_wait(20) # seconds
 driver.get("Your-URL")
 myElement = driver.find_element_by_id("myElement")

You can use any of above. Both are valid.

Helping Hands
  • 5,292
  • 9
  • 60
  • 127
  • 1
    perfect that,s what i need, thank you. Just to clarify, explicit wait will applied for that element only while implicit wait will apply for the whole session, am i correct? – Kiddo Dec 22 '14 at 10:42
  • Yes you are right , Still if you want main diff. between them then please refer : http://stackoverflow.com/questions/22656615/what-is-difference-between-implicit-wait-vs-explicit-wait-in-selenium-webdriver – Helping Hands Dec 22 '14 at 10:59
  • 2
    @HelpingHands : Both of the above waits you have used are infact [**`Explicitwaits`**](http://docs.seleniumhq.org/docs/04_webdriver_advanced.jsp#explicit-waits), not [**`Implicit waits`**](http://docs.seleniumhq.org/docs/04_webdriver_advanced.jsp#implicit-waits). Please check and edit your code accordingly. – Subh Dec 22 '14 at 12:39
9

You need to use Selenium Waits.

In particular, element_to_be_clickable expected condition is what fits better than others:

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

element = WebDriverWait(driver, 10).until(
    EC.element_to_be_clickable((By.ID, "myDynamicElement"))
)
element.click()

where driver is your webdriver instance, 10 is the amount of seconds to wait for an element. With this setup, selenium would try to locate an element every 500 milliseconds for 10 seconds in total. It would throw TimeoutException after 10 seconds left if element would not be found.

alecxe
  • 462,703
  • 120
  • 1,088
  • 1,195