How to read JSON from URL in Python?

Question

I am trying to use Python to get a JSON file from the Web. If I open the URL in my browser (Mozilla or Chromium) I do see the JSON. But when I do the following with the Python:

response = urllib2.urlopen(url)
data = json.loads(response.read())

I get an error message that tells me the following (after translation in English): Errno 10060, a connection troughs an error, since the server after a certain time period did not react, or the connection was erroneous, or the host did not react.

ADDED

It looks like there are many people who faced the described problem. There are also some answers to the similar (or the same) question. For example here we can see the following solution:

import requests

r = requests.get("http://www.google.com", proxies={"http": "http://61.233.25.166:80"})
print(r.text)

It is already a step forward for me (I think that it is very likely that the proxy is the reason of the problem). However, I still did not get it done since I do not know URL of my proxy and I probably will need user name and password. Howe can I find them? How did it happen that my browsers have them I do not?

ADDED 2

I think I am now one step further. I have used this site to find out what my proxy is: http://www.whatismyproxy.com/

Then I have used the following code:

proxies = {'http':'my_proxy.blabla.com/'}
r = requests.get(url, proxies = proxies)
print r

As a result I get

<Response [404]>

Looks not so good, but at least I think that my proxy is correct, because when I randomly change the address of the proxy I get another error:

Cannot connect to proxy

So, I can connect to proxy but something is not found.

Answer 1

I think there might be something wrong, when you're trying to get the json from the online source(URL). Just to make things clear, here is a small code snippet

#!/usr/bin/env python

try:
    # For Python 3+
    from urllib.request import urlopen
except ImportError:
    # For Python 2
    from urllib2 import urlopen

import json

def get_jsonparsed_data(url):
    response = urlopen(url)
    data = str(response.read())
    return json.loads(data)

If you still get a connection error, You can try a couple of steps:

1. Try to urlopen() a random site from the Interpreter (Interactive Mode). If you are able to grab the source code you're good. If not check internet conditions or try the request module. Check here

2. Check and see if the json in the URL is in the correct syntax. For sample json syntax check here

3. Try the simplejson module.

Edit 1: if you want to access websites using a system wide proxy you will have to use a proxy handler to use loopback(local host) to connect to that proxy.. A sample code is shown below.

proxy = urllib2.ProxyHandler({
    'http': '127.0.0.1',
    'https': '127.0.0.1'
})
opener = urllib2.build_opener(proxy)
urllib2.install_opener(opener)
# this way you can send both http and https request using proxies
urllib2.urlopen('http://www.google.com')
urllib2.urlopen('https://www.google.com')

I have not not worked a lot with ProxyHandler. I just know the theory and code. I am sure there are better ways to access websites through proxies; One which does not involve installing the opener everytime you run the program. But hopefully it will point you in the right direction.