3

In python if I import requests and do:

t = requests.get("http://www.azlyrics.com/u/urban.html")

I get this exception:

raise BadStatusLine(line)
http.client.BadStatusLine: ''

Does anyone know how to fix this?

Honinbo Shusaku
  • 1,411
  • 2
  • 27
  • 45
OultimoCoder
  • 244
  • 2
  • 7
  • 24

1 Answers1

7

There can be different reasons for this kind of error, but in this particular case this looks like a simple web-scraping detection - it can be solved by providing a User-Agent header pretending to be a real browser:

In [2]: requests.get("http://www.azlyrics.com/u/urban.html")
...
ConnectionError: ('Connection aborted.', BadStatusLine("''",))

In [3]: requests.get("http://www.azlyrics.com/u/urban.html", headers={'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.75 Safari/537.36'})
Out[3]: <Response [200]>
Community
  • 1
  • 1
alecxe
  • 462,703
  • 120
  • 1,088
  • 1,195
  • I forget that I have adjusted nginx to refuse "bad user agents". Now I was bad. Thanks a lot. ( orginal code I wrote in php, I couldnt find solution then moved whole project to python for that stupid self feature ) – nerkn Nov 09 '17 at 13:23