Downloading CSV Online and Converting to Pandas DF

Question

I have a CSV file that I'm downloading online. I'm trying to get it into a Pandas DataFrame. The source is the Federal Reserve Bank and using inflation data (CPI). I'm missing something to make this work.

Here's the URL to download:

s_url = https://fred.stlouisfed.org/graph/fredgraph.csv?cosd=1990-01-01&coed=2040-01-01&id=CPIAUCSL

I'm taking that URL and reading it with this:

response = urlopen(s_url).read()

That gives me something that looks like this:

b'DATE,CPIAUCSL\n1990-01-01,127.5\n1990-02-01,128.0\n1990-03-01,128.6\n1990-04-01,128.9\n1990-05-01,129.1\n1990-06-01,129.9\n1990-07-01,130.5\n1990-08-01,131.6\n1990-09-01,132.5\n1990-10-01,133.4\n1990-11-01,133.7\n1990-12-01,134.2\n1991-01-01,134.7\n1991-02-01,134.8\n1991-03-01,134.8\n1991-04-01,135.1\n1991-05-01,135.6\n1991-06-01,136.0\n'

I tried to then take that response and read it into Pandas:

df = pd.read_csv(response)

That doesn't work. What steps am I missing in this process?

Does this answer your question? [Pandas read\_csv from url](https://stackoverflow.com/questions/32400867/pandas-read-csv-from-url) — Soumendra Mishra, Sep 14 '20 at 12:24

score 2 · Accepted Answer · answered Sep 14 '20 at 12:04

2

It seem that using url directly as df argument works. Did you try it?

import pandas as pd 
s_url = 'https://fred.stlouisfed.org/graph/fredgraph.csv?cosd=1990-01-01&coed=2040-01-01&id=CPIAUCSL' 
df = pd.read_csv(s_url)

answered Sep 14 '20 at 12:04

RunTheGauntlet

392
1
4
15

1

No I didn't. What a fool I feel like right now, but thanks for the answer! – Ragnar Lothbrok Sep 14 '20 at 12:09

score 1 · Answer 2 · answered Sep 14 '20 at 12:06

You can use stringIO to read data from string.

data = b'DATE,CPIAUCSL\n1990-01-01,127.5\n1990-02-01,128.0\n1990-03-01,128.6\n1990-04-01,128.9\n1990-05-01,129.1\n1990-06-01,129.9\n1990-07-01,130.5\n1990-08-01,131.6\n1990-09-01,132.5\n1990-10-01,133.4\n1990-11-01,133.7\n1990-12-01,134.2\n1991-01-01,134.7\n1991-02-01,134.8\n1991-03-01,134.8\n1991-04-01,135.1\n1991-05-01,135.6\n1991-06-01,136.0\n'
from io import StringIO
import pandas as pd
data = data.decode('utf-8') #decode binary string to utf-8
df = pd.read_csv(StringIO(data)) #read the data
df

Output:

DATE    CPIAUCSL
0   1990-01-01  127.5
1   1990-02-01  128.0
2   1990-03-01  128.6
3   1990-04-01  128.9
4   1990-05-01  129.1
5   1990-06-01  129.9

Ah, this is technically the answer I was looking for. I knew I was missing something. However, I didn't realize I could just directly plug the URL into pd.read_csv(). — Ragnar Lothbrok, Sep 14 '20 at 12:27

Downloading CSV Online and Converting to Pandas DF

2 Answers2