Urllib2 not working, http forbidden error

Question

I'm trying to use the yts api, and I wrote the following code.

import urllib2
import json

url = 'https://yts.ag/api/v2/list_movies.json?quality=3D'

json_obj = urllib2.urlopen(url)

data = json.load(json_obj)

print data

But I ran into the following error:

File "<stdin>", line 1, in <module>
  File "C:\Python27\lib\urllib2.py", line 154, in urlopen
    return opener.open(url, data, timeout)
  File "C:\Python27\lib\urllib2.py", line 435, in open
    response = meth(req, response)
  File "C:\Python27\lib\urllib2.py", line 548, in http_response
    'http', request, response, code, msg, hdrs)
  File "C:\Python27\lib\urllib2.py", line 473, in error
    return self._call_chain(*args)
  File "C:\Python27\lib\urllib2.py", line 407, in _call_chain
    result = func(*args)
  File "C:\Python27\lib\urllib2.py", line 556, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 403: Forbidden

Help me how to fix this

Answer 1

I'm not completely sure what's going on with your urllib2 request, as it works fine with the other urls I tested, with and without JSON responses.

I suspect it's an encoding thing for the quality=3D query, but my brief attempt at handling that also failed.

Simply putting the request in a browser works, as does the following 'requests' code.

In a nutshell, I'm not truly answering your question regarding fixing the urllib2 usage, but definitely fixing the problem of getting the data.

I'd suggest working with the 'requests' module in this case. Try

import requests
import json

url = 'https://yts.ag/api/v2/list_movies.json?quality=3D'

response = requests.get( url = url )

data = response.json()

print data

Note that the requests response.json() handles the json.load() for you.

It will save you many a headache over urllib2.

The 'requests' documentation: http://docs.python-requests.org/en/latest/