Python Requests HTTP Response 406

Question

I have my own domain where an json file is stored (http://example.com/file.json). When accesing the file in browser using the direct link, the json is returned just fine. But when using the same approach in my python code below, the http response is 406. Any ideas why?

import requests
url = 'http://example.com/file.json'
r = requests.get(url, headers={"Accept":"text/html"})
print(r.status_code)
print(r.headers)

Prints:

406
{'Server': 'nginx/1.14.1', 'Date': 'Sun, 12 May 2019 16:53:25 GMT', 'Content-Type': 'text/html; charset=iso-8859-1', 'Content-Length': '226', 'Connection': 'k
eep-alive'}

Sounds like you might need to add an `accept` header to the request. — Mark, May 12 '19 at 17:21
@MarkMeyer do you have any example, please? I don't see anything like that in the doc. https://2.python-requests.org//en/latest/api/ — DNac, May 12 '19 at 18:49
@Mark: The Requests module includes "accept */*" by default. — Oddthinking, Aug 26 '21 at 01:22

score 55 · Accepted Answer · edited Feb 05 '20 at 16:10

55

Solved by using a different User-Agent. The default Python User-Agent 'python-requests/2.21.0' was being probably blocked by the hosting company.

r = requests.get(url, headers={"User-Agent": "XY"})

Some of the possible agents: List of User Agent strings

edited Feb 05 '20 at 16:10

Laurent Bristiel

6,819
34
52

answered May 13 '19 at 13:13

DNac

2,663
8
31
54

also, if you'd just like to use the UA sent out by your browser, you can get it here: https://www.whatismybrowser.com/detect/what-is-my-user-agent – Anupam Nov 09 '20 at 11:10

score -1 · Answer 2 · answered May 05 '21 at 13:01

-1

Your own accepted answer is most likely incorrect. This error means the value of Accept header does not match Content-Type of the response. If you'd send Accept: */* , it would work (at least the 406 would not be returned).

answered May 05 '21 at 13:01

Krzysztof Szularz

5,151
24
35

I agree, this makes more sense... but I just encountered this same problem, and checked, and "Accept: */*" was included (by default) in the request header. Further, changing the User-Agent as suggested worked. Weird that my (big player) web-host bans Python Requests by default. – Oddthinking Aug 26 '21 at 01:21
While that's the _classic_ 406 behavior you'd expect, it turns out some hosts block requests made with the default User-Agent while returning a 406, as @DNac pointed out. See this other question for another example: https://stackoverflow.com/questions/57417799/i-call-api-from-python-i-get-the-response-406-not-acceptable – arredond Mar 01 '22 at 14:01

Python Requests HTTP Response 406

2 Answers2

Linked