Saving a downloaded CSV file using Python

Question

I want to download a csv file from a link with request and save it as MSFT.csv. However, my code return error

File "< stdin >", line 1, in _csv.Error: new-line character seen in unquoted field - do you need to open the file in universal-newline mode?

import requests
import csv

data=requests.get('https://www.alphavantage.co/query?function=TIME_SERIES_DAILY_ADJUSTED&symbol=MSFT&apikey=demo&datatype=csv'
cr = csv.reader(data)

for row in cr:
    print row

How can I save it with MSFT.csv?

cs95 · Accepted Answer · 2019-06-19T21:20:11.367

27

If you're trying to write this data to a CSV file, you can first download it using requests.get, then save each line to a CSV file.

import csv
import requests

url = 'https://www.alphavantage.co/query?function=TIME_SERIES_DAILY_ADJUSTED&symbol=MSFT&apikey=demo&datatype=csv'
response = requests.get(url)        

with open('out.csv', 'w') as f:
    writer = csv.writer(f)
    for line in response.iter_lines():
        writer.writerow(line.decode('utf-8').split(','))

Alternatively, if you have pandas installed (pip install --user pandas), you can load data by passing a URL directly.

import pandas as pd

df = pd.read_csv(url)   
df.head()

    timestamp    open    high     low   close  adjusted_close    volume  dividend_amount  split_coefficient
0  2019-06-19  135.00  135.93  133.81  135.69          135.69  17946556              0.0                1.0
1  2019-06-18  134.19  135.24  133.57  135.16          135.16  25908534              0.0                1.0
2  2019-06-17  132.63  133.73  132.53  132.85          132.85  14517785              0.0                1.0
3  2019-06-14  132.26  133.79  131.64  132.45          132.45  17821703              0.0                1.0
4  2019-06-13  131.98  132.67  131.56  132.32          132.32  17200848              0.0                1.0

df.to_csv('out.csv')

edited Jun 19 '19 at 21:20

answered Aug 31 '17 at 09:54

cs95

379,657
97
704
746

The code that uses `import csv` worked perfect for me. The one with pandas is not that perfect when compared to the downloaded file with a web browser .. differences of few bytes may be. – eddys Mar 04 '18 at 13:14
@eddys I see... try passing the url string to read_csv directly. That may work better. – cs95 Mar 04 '18 at 15:31
I think if the task is copying the csv as it is, I would recommend df.to_csv('out.csv',index=False). Otherwise there will be an extra column in out.csv. – J_Scholz Jan 30 '20 at 19:59
@J_Scholz That's assuming you don't want the index (which is true in OP's specific case but not always in general). Either drop the index when saving, or use `index_col=[0]` when loading. – cs95 Jan 30 '20 at 20:08
There a new line extra when tried to implement it, Each row had a null row inbetween, Something to rectify that? – Dr.House Aug 18 '22 at 11:51

score 8 · Answer 2 · answered Aug 31 '17 at 10:13

You can achieve it via requests as

import os
import requests

def download_file(url, filename):
    ''' Downloads file from the url and save it as filename '''
    # check if file already exists
    if not os.path.isfile(filename):
        print('Downloading File')
        response = requests.get(url)
        # Check if the response is ok (200)
        if response.status_code == 200:
            # Open file and write the content
            with open(filename, 'wb') as file:
                # A chunk of 128 bytes
                for chunk in response:
                    file.write(chunk)
    else:
        print('File exists')

You can call the function with your url and filename that you want. In your case it would be:

url = 'https://www.alphavantage.co/query?function=TIME_SERIES_DAILY_ADJUSTED&symbol=MSFT&apikey=demo&datatype=csv'
filename = 'MSFT.csv'
download_file(url, filename)

Hope this helps.

score 0 · Answer 3 · answered Oct 05 '18 at 19:32

0

There is a simpler way for you.

import urllib.request

csv_url = 'https://www.alphavantage.co/query?function=TIME_SERIES_DAILY_ADJUSTED&symbol=MSFT&apikey=demo&datatype=csv'

urllib.request.urlretrieve(csv_url, 'MSFT.csv')

answered Oct 05 '18 at 19:32

Ahmed AbdelKhalek

184
1
11

score -1 · Answer 4 · answered Aug 31 '17 at 09:57

Here you go

import requests, csv

download = requests.get('https://www.alphavantage.co/query?function=TIME_SERIES_DAILY_ADJUSTED&symbol=MSFT&apikey=demo&datatype=csv')

with open('MSFT.csv', 'w') as temp_file:
    temp_file.writelines(download.content)

Saving a downloaded CSV file using Python

4 Answers4

Linked