wanna have a script that scrapes the titles of a list of URLs, but it could be super slow if we need to wait until the whole page gets loaded. The title is the only thing I am looking for.
Can we stop page loading when the title gets loaded? maybe with something like EC.title_contains.
Have you tried with setting up pageLoadStrategy in your selenium capabilities ?
You can read more about it in this thread - Page load strategy for Chrome driver
and
http://webdriver.io/guide/getstarted/configuration.html - a bit outdated which says it is available only for firefox but should be available for all browsers by now.
The problem is that webdriver.io as example waits until the page has fully loaded and the loading timer in the tab is away. This is for a good reason because a lot of API´s like .getText are not working until the complete page is loaded because sometimes the element will only be loaded at the end as example. But you can reduce the loading time by:
1. You use extension like script safe or other simple script blocker that block EVERYTHING with javascript inline or external.
2. Go to chrome settings and disable everything like cookies, javascript, flash etc. just everything.
3. Go to chrome://flags and disable everything from javascript (all API´s like gamepad API ETC.) to WebGL, Canvas etc. - You can really disable everything I also have a chrome profile where I disabled everything.
Now with normal Internet Speed and good CPU you can open every site in 1-3 seconds.
Or alternative you can try a headless browser.
