0

I'm writing a parser and I'm using Selenium Webdriver. So, I have this https://repl.it/Dgtp code and it's working fine until one random element and throws following exception: http.client.BadStatusLine: ''
Don't know how to fix it at all. Help.

[UPD]

I tried to scroll the page via webdriver (it had to cause thumbnails to load) and got this https://repl.it/DkiX error series. It would be caused by HTTP error from one of images which were loading, but I've not found any loading errors on the page. Still searching the answer.

ndawn
  • 1
  • 2

1 Answers1

0

This is a urllib problem. This happens most commonly in python3. What it means is that the status code that the server returned isn't recognized by the http library. Sometimes the server doesn't receive the request at all and the status code returns an empty string triggering that error.

How to fix

Check if the URL string has a trailing newline character. Make sure that your URLs are stripped of any leading or trailing special characters. More info here

If everything looks fine with the URL just treat the exception

import http
try:
    browser.get(MONTHLY_URL)
except http.client.HTTPException as e:
    print e
Community
  • 1
  • 1
Rafael Almeida
  • 5,142
  • 2
  • 20
  • 33
  • Excuse me, but I have only one URL in this Python script. It's http://previewsworld.com/catalog and it doesn't seems to be reason of my error (or I just don't understand your fix suggestion). I don't even have any errors in browser.get() part (I tried to handle this error as you wrote, but it doesn't make a difference). I'm getting BadStatusLine when I'm trying to parse already given HTML and I completely don't know what to do with this. – ndawn Sep 24 '16 at 23:37
  • Update your question with the line that is triggering the error. – Rafael Almeida Sep 24 '16 at 23:42
  • Updated. I tried to wrap entry creating procedure in try-except construction and noticed that it throws different errors: sometimes it raises https://repl.it/Dgw5 , sometimes - https://repl.it/Dgw5/1 and sometimes it throws familiar BadStatusLine error. Can't explain this phenomenon, server which runs this script doesn't have any connection problems and there is actually only one connection attempt, I don't manipulating any network things after getting connected to MONTHLY_URL. – ndawn Sep 25 '16 at 00:18
  • Updated once more. – ndawn Sep 29 '16 at 18:30