https get request with python urllib2

Question

I am trying to fetch data from quandl using urllib2.Please check code below.

import json
from pymongo import MongoClient
import urllib2
import requests
import ssl
#import quandl
codes = [100526];
for id in codes:
    url = 'https://www.quandl.com.com//api/v3/datasets/AMFI/"+str(id)+".json?api_key=XXXXXXXX&start_date=2013-08-30'
    req = urllib2.Request(url)
    response = urllib2.urlopen(req)
    data = response.read()
    print data

OR

for id in codes:
    url = "https://www.quandl.com.com//api/v3/datasets/AMFI/"+str(id)+".json?api_key=XXXXXXXX&start_date=2013-08-30"
    request = requests.get(url,verify=False)
    print request

I am getting HTTPERROR exception 404 in 1st case. and when I use request module I get SSL error even after using verify=false. I am looking through previous posts but most of them are related to HTTP request.

Thanks for help. J

Emil Rowland · Answer 1 · 2017-08-26T09:24:03.097

2

This is working for me, but you get a warning about the SSL certificate but you don't need to care about it.

import requests
codes = [100526];
for id in codes:
    url = "https://www.quandl.com.com//api/v3/datasets/AMFI/"+str(id)+".json?api_key=XXXXXXXX&start_date=2013-08-30"
    request = requests.get(url, verify=False)
    print request.text

request.text has your response data.

edited Aug 26 '17 at 09:24

answered Aug 26 '17 at 09:14

Emil Rowland

538
5
25

score 1 · Accepted Answer · answered Aug 26 '17 at 09:33

You seem to be using a wrong URL (.com.com instead of .com) as well as a combination of different quotes in the first version of your code. Use the following instead and it should work:

import urllib2
import requests

codes = [100526]

for id in codes:
    url = "https://www.quandl.com//api/v3/datasets/AMFI/"+str(id)+".json?start_date=2013-08-30"
    req = urllib2.Request(url)
    response = urllib2.urlopen(req)
    print response.read()

for id in codes:
    url = "https://www.quandl.com//api/v3/datasets/AMFI/"+str(id)+".json?start_date=2013-08-30"
    response = requests.get(url,verify=False)
    print response.text

To disable the warning about the SSL certificate, use the following code before making the request using requests:

from requests.packages.urllib3.exceptions import InsecureRequestWarning
requests.packages.urllib3.disable_warnings(InsecureRequestWarning)

Thanks for the response. I guess .com was causing error. And to be sure I have used `request = requests.get(url,cert=('C:\\ssl\\ca.crt', 'C:\\ssl\\ca.key')) jsonReq = request.json()` — Jitesh, Aug 26 '17 at 11:50

https get request with python urllib2

2 Answers2