236

I'm using the Requests library and accessing a website to gather data from it with the following code: 

r = requests.get(url)

I want to add error testing for when an improper URL is entered and a 404 error is returned. If I intentionally enter an invalid URL, when I do this:

print r

I get this:

<Response [404]>

EDIT:

I want to know how to test for that. The object type is still the same. When I do r.content or r.text, I simply get the HTML of a custom 404 page.

Manish Gupta
  • 4,438
  • 18
  • 57
  • 104
user1427661
  • 11,158
  • 28
  • 90
  • 132
  • 4
    Look at the documentation: http://docs.python-requests.org/en/latest/ The first page states to look at r.status_code – Udo Klein Mar 06 '13 at 21:50

2 Answers2

470

Look at the r.status_code attribute:

if r.status_code == 404:
    # A 404 was issued.

Demo:

>>> import requests
>>> r = requests.get('http://httpbin.org/status/404')
>>> r.status_code
404

If you want requests to raise an exception for error codes (4xx or 5xx), call r.raise_for_status():

>>> r = requests.get('http://httpbin.org/status/404')
>>> r.raise_for_status()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "requests/models.py", line 664, in raise_for_status
    raise http_error
requests.exceptions.HTTPError: 404 Client Error: NOT FOUND
>>> r = requests.get('http://httpbin.org/status/200')
>>> r.raise_for_status()
>>> # no exception raised.

You can also test the response object in a boolean context; if the status code is not an error code (4xx or 5xx), it is considered ‘true’:

if r:
    # successful response

If you want to be more explicit, use if r.ok:.

Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
  • 2
    Well damn that's annoying. I wish there was a way to get requests.get to raise for any reason except a 2XX without having to make extra calls. – Asfand Qazi Jan 31 '19 at 12:12
  • 1
    @AsfandQazi: why would you want to raise an exception for a 1xx or 3xx status code? Those are not error conditions. – Martijn Pieters Jan 31 '19 at 12:26
  • 2
    1xx and 3xx should (and are) handled transparently by the library. I was talking specifically about 4xx errors. Of course sometimes you may want to read a 4xx response so we shouldn't impose it on everyone, but I wish there was a flag you could pass to `.get()` instead of calling a function afterwards. – Asfand Qazi Jan 31 '19 at 12:44
  • @AsfandQazi: it's trivial to write your own wrapper function if that's important to you. – Martijn Pieters Jan 31 '19 at 12:45
  • 3
    It just seems a bit untidy, that's all – Asfand Qazi Jan 31 '19 at 14:40
6

If your request is made inside another function, but you want to catch the error in a higher level, it is good to know that you can also get the status code directly from the exception. In my case I could not access the response since the HTTPError was raised before my function was able to pass on the response. I ended up doing the following:

try:
     r = function_calling_request(the_request)
except HTTPError as e:
     if e.response.status_code == 404:
          return do_stuff_if_not_found()
Boketto
  • 707
  • 7
  • 17