14

I'm using PhantomJS as a webdriver to load some urls. Usually, the program runs fine. However, it hangs on driver.get(url) a lot, and i'm wondering if there is anything I can do about it? 

driver = webdriver.PhantomJS(executable_path= path_to_phantomjs_exe, service_log_path= path_to_ghostdriver_log)
driver.get(url)

It will just hang trying to load a certain url forever. But if i try it again, it might work. Are webdrivers/phantomJS really just that unstable? I guess last resort would be to constantly call driver.get(url) until it finally loads, but is that really going to be necessary? Thanks!

EDIT: It seems to only hang when loading the first link out of a list of them. It eventually does load, however, but after a few minutes. The rest of the links load within seconds. Any help at all would be great.

James Lemieux
  • 720
  • 1
  • 9
  • 26
  • Did you try using Chrome or Firefox, and did you get the same issue? – Louis Jan 24 '15 at 23:10
  • Yes, and the pages load almost instantly, so it is not an issue of a long loading page. PhantomJS just seems to have a hard time i guess... any ideas on how to diagnose/debug/fix? – James Lemieux Jan 25 '15 at 01:02
  • I assume you have the latest phantomjs and you have tried reinstalling it? – nilesh Jan 25 '15 at 17:56
  • Yes that is correct. It did not use to do this. Everything always ran smoothly, now all of a sudden it takes minutes to load the first url. Everytime I interrupt it, the last call is `data = self._sock.recv(self._rbufsize)`, which im guessing is what it's hanging on. Don't know if that helps. **edit:** Also, it works fine on my mac, which is on the same connection. – James Lemieux Jan 25 '15 at 19:57
  • I have similar problems (see http://stackoverflow.com/questions/29108260/geb-selenium-tests-hang-loading-new-page) - what do you see if you pass the "--debug=true" option to the phantomjs command line? – wrschneider Mar 23 '15 at 01:59
  • @wrschneider99 The problem actually just... went away. And I have no way of recreating it. It had only hung sometimes... and other times it worked fine. There really was no explanation to it from my perspective, so I wish the best of luck to you. – James Lemieux Mar 24 '15 at 02:32

2 Answers2

10

I've answered this exact problem on this post here: Geb/Selenium tests hang loading new page but copied it here because I see that this question is older.

I hope you can find a way to implement this into your code, but this is what worked for me when I was having a similar situation with PhantomJS hanging.

I traced it to be hanging on a driver.get() call, which for me was saying something wasn't going through or the webdriver just wasn't - for some reason - giving the load successful command back to the driver, allowing the script to continue.

So, I added the following:

driver = webdriver.PhantomJS()

# set timeout information
driver.set_page_load_timeout(15)

I've tested this at a time of 5 (seconds) and it just didn't wait long enough and nothing would happen. 15 seconds worked great for me, but that's maybe something you should test.

On top of this, I also created a loop whenever there was an option for the webdriver to timeout, so that the driver.get() could attempt to re-send the .get() command. Implementing a try / except stacked scenario, I was able to approach this:

while finished == 0:
    try:
        driver.get(url3)
        finished = 1
    except:
        sleep(5)

I have seen an except handle as:

except TimeoutException as e:
    #Handle your exception here
    print(e)

but I had no use for this. It might be nice to know how to catch specific exceptions, though.

See this solution for more options for a timeout: Setting timeout on selenium webdriver.PhantomJS

Community
  • 1
  • 1
ntk4
  • 1,247
  • 1
  • 13
  • 18
3

So I was having the same problem:

driver = webdriver.PhantomJS(executable_path= path_to_phantomjs_exe, service_log_path= path_to_ghostdriver_log)
driver.get(url)

So I changed the service_log_path to:

service_log_path=os.path.devnull

This seemed to work for me!!!