1

There is a lot that has been written about timeout and selenium and page loads.

But almost none of it works in chromedriver.

And all of what works is not exactly what I am looking for.

Note: I am not looking for set_page_load_timeout()

What do I want:

  • I say: driver.get("some-weird-slow-place")
  • chromedriver says: yes, yes... on my way
  • [15 seg later...] still on my way
  • [20 seg later...] okay sir... please do javascript window.stop();
  • But! Keep working as usual with whatever loaded elements you have.

Why I want this:

Because maybe I just want to get the url of the site and its title... and not the fancy huge background image or the crunchy brunchy punchy animated banners and multiple thousand jquery magics that are still loading.

What did I try:

driver.get(url)
driver.execute_script("setInterval(function(){ window.stop(); }, 20000);")

But it does not work, because driver.get() will wait until page is loaded before executing the script.

Álvaro N. Franz
  • 1,188
  • 3
  • 17
  • 39
  • This may help: https://stackoverflow.com/questions/11455210/selenium-webdriver-go-to-page-without-waiting-for-page-load – Deem Jul 26 '17 at 08:25
  • @Windmill I saw that already but it says "This method fails in any browser other than Firefox" and I am forced to use chromedriver. – Álvaro N. Franz Jul 26 '17 at 08:31

1 Answers1

0

Why do you need selenium for this ? How about https://www.crummy.com/software/BeautifulSoup/bs4/doc/

  • 2
    Would have been better as a comment. – Himal Jul 26 '17 at 08:28
  • Because this is not the sole purpose of me using selenium. It is just a little step in a bigger scrapper, that sometimes needs to execute actions that require Selenium, and other times just needs to grab other elements for which waiting for the whole page load is a waste of time. – Álvaro N. Franz Jul 26 '17 at 08:30
  • I see, but i guess beatifulsoup is fastest and easiest way to grab page title, than selenium. You can easily combine 2 instruments to achive better performance. If you wish to stick to selenium, you can try to call JS in separated python thread ( not sure it work, just as idea) https://stackoverflow.com/questions/2846653/how-to-use-threading-in-python –  Jul 26 '17 at 08:42
  • I understand your point of view, and I accept that using both instruments would be the pro approach. I will consider it for other cases but now I am forced to stick to selenium. I do not know the url i want to open. Because the website opens after a click on an element that calls a javascript that calls an external server and throws back the link. – Álvaro N. Franz Jul 26 '17 at 08:49