500 error with urllib.request.urlopen

Question

The following code:

   req = urllib.request.Request(url=r"http://borel.slu.edu/cgi-bin/cc.cgi?foirm_ionchur=im&foirm=Seol&hits=1&format=xml",headers={'User-Agent':' Mozilla/5.0 (Windows NT 6.1; WOW64; rv:12.0) Gecko/20100101 Firefox/12.0'})
   handler = urllib.request.urlopen(req)

is giving me the following exception:

Traceback (most recent call last):
  File "C:/Users/Foo/lang/old/test.py", line 46, in <module>
    rip()
  File "C:/Users/Foo/lang/old/test.py", line 36, in rip
    handler = urllib.request.urlopen(req)
  File "C:\Python32\lib\urllib\request.py", line 138, in urlopen
    return opener.open(url, data, timeout)
  File "C:\Python32\lib\urllib\request.py", line 375, in open
    response = meth(req, response)
  File "C:\Python32\lib\urllib\request.py", line 487, in http_response
    'http', request, response, code, msg, hdrs)
  File "C:\Python32\lib\urllib\request.py", line 413, in error
    return self._call_chain(*args)
  File "C:\Python32\lib\urllib\request.py", line 347, in _call_chain
    result = func(*args)
  File "C:\Python32\lib\urllib\request.py", line 495, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 500: Internal Server Error

but it works fine in my browser, whats the issue?

Interestingly enough, if you catch the exception and call `.read()` on that, you get an XML document that looks like a successful response. — Martijn Pieters, Apr 09 '13 at 20:55
It just always replies with status 500, in browsers as well. — Pavel Anossov, Apr 09 '13 at 20:57

Martijn Pieters · Answer 1 · 2013-04-09T21:12:03.233

19

The server is rather b0rken. It responds with a 500 error in the browser as well.

You can catch the exception and still read the response:

import urllib.request
from urllib.error import HTTPError

req = urllib.request.Request(url=r"http://borel.slu.edu/cgi-bin/cc.cgi?foirm_ionchur=im&foirm=Seol&hits=1&format=xml",headers={'User-Agent':' Mozilla/5.0 (Windows NT 6.1; WOW64; rv:12.0) Gecko/20100101 Firefox/12.0'})
try:
    handler = urllib.request.urlopen(req)
except HTTPError as e:
    content = e.read()

edited Apr 09 '13 at 21:12

answered Apr 09 '13 at 21:00

Martijn Pieters

1,048,767
296
4,058
3,343

2

If you're like me and got an error about urllib.error not being defined, and you're actually using urllib2, try `from urllib2 import HTTPError` instead – streetlogics Feb 26 '14 at 23:11

score 1 · Answer 2 · answered Sep 11 '21 at 21:36

1

When it happened to me I've reduced the plt.figure size parameter and it worked. It may be some odd parameter on your code that is not being able to be read.

answered Sep 11 '21 at 21:36

Renato Melo

51
6

500 error with urllib.request.urlopen

2 Answers2

Linked