Downloading a csv file with python

Question

I'm trying to download historical stock prices from Yahoo Finance using Python using the following code:

import urllib.request
import ssl
import os

url = 'https://query1.finance.yahoo.com/v7/finance/download/%5ENSEI?period1=1537097203&period2=1568633203&interval=1d&events=history&crumb=0PVssBOEZBk'

ctx = ssl.create_default_context()
ctx.check_hostname = False
ctx.verify_mode = ssl.CERT_NONE

connection = urllib.request.urlopen(url,context = ctx)
data = connection.read()

with urllib.request.urlopen(url) as testfile, open('data.csv', 'w') as f:
    f.write(testfile.read().decode())

however, I'm getting a traceback as mentioned below:

Traceback (most recent call last):
  File "C:/Users/jmirand/Desktop/test.py", line 11, in <module>
    connection = urllib.request.urlopen(url,context = ctx)
  File "C:\Users\jmirand\AppData\Local\Programs\Python\Python37-32\lib\urllib\request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "C:\Users\jmirand\AppData\Local\Programs\Python\Python37-32\lib\urllib\request.py", line 531, in open
    response = meth(req, response)
  File "C:\Users\jmirand\AppData\Local\Programs\Python\Python37-32\lib\urllib\request.py", line 641, in http_response
    'http', request, response, code, msg, hdrs)
  File "C:\Users\jmirand\AppData\Local\Programs\Python\Python37-32\lib\urllib\request.py", line 569, in error
    return self._call_chain(*args)
  File "C:\Users\jmirand\AppData\Local\Programs\Python\Python37-32\lib\urllib\request.py", line 503, in _call_chain
    result = func(*args)
  File "C:\Users\jmirand\AppData\Local\Programs\Python\Python37-32\lib\urllib\request.py", line 649, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 401: Unauthorized

I assume this has to do with the fact that it's because of the HTTPS and Python doesn't have enough certificates to put into it by default.

The webpage im on is here Yahoo Finance NSEI historical prices and its on the 'Download Data' tab that you click on where the data automatically gets downloaded through a csv file.

Can you please help in rectifying the code?

score 0 · Accepted Answer · answered Sep 16 '19 at 12:19

The yahoo api expects cookies from your browser to authenticate. I have copied the cookies from my browser and passed them through python requests

import requests
import csv


url = "https://query1.finance.yahoo.com/v7/finance/download/%5ENSEI?period1=1537099135&period2=1568635135&interval=1d&events=history&crumb=MMDwV5mvf2J"


cookies = {
            "APID":"UP26c2bef4-bc0b-11e9-936a-066776ea83e8",
            "APIDTS":"1568635136",
            "B":"d10v5dhekvhhg&b=3&s=m6",
            "GUC":"AQEBAQFda2VeMUIeqgS6&s=AQAAAICdZvpJ&g=XWoXdg",
            "PRF":"t%3D%255ENSEI",
            "cmp":"t=1568635133&j=0",
            "thamba":"2"
    }
with requests.Session() as s:
    download = s.get(url,cookies=cookies)

    decoded_content = download.content.decode('utf-8')

    cr = csv.reader(decoded_content.splitlines(), delimiter=',')
    my_list = list(cr)
    for row in my_list:
        print(row)

Thank you, this helps. Can you please help in exporting the file to a csv. Don't think this downloads the file. — Marzipan, Sep 16 '19 at 17:54
Replace the reader part with a writer. Refer to this https://stackoverflow.com/questions/45978295/saving-a-downloaded-csv-file-using-python — Harrison, Sep 17 '19 at 06:17

Downloading a csv file with python

1 Answers1