Creating a list of dictionaries from a URL that points to a CSV

Question

Hello and thanks in advance!

I have a url link such as: https://datasets/file.csv. The csv file it points to looks like this:

a, b, c, d
1, 2, 3, 4
5, 6, 7, 8

From this data I need to produce a list of dictionaries such as:

[{'a': '1', 'b': '2', 'c': '3', 'd': '4'},
 {'a': '5', 'b': '6', 'c': '7', 'd': '8'}]

I've tried this method:

from csv import DictReader
link = "https://datasets/file.csv"
with open(link, 'r') as read_obj:
     dict_reader = DictReader(read_obj)
     list_of_dict = list(dict_reader)
     print(list_of_dict)

However when I pass in my link, I get an invalid argument error. I'm assuming:

with open("is expecting a file object", 'r') as read_obj:

Should I be converting my url link to some other type/object or is there just a better way of doing this? Thanks again!

Does this answer your question? [How can I read the contents of an URL with Python?](https://stackoverflow.com/questions/15138614/how-can-i-read-the-contents-of-an-url-with-python) — esqew, Oct 03 '20 at 18:33

score 0 · Accepted Answer · answered Oct 03 '20 at 18:37

get the csv from url using requests.get
Read the csv into pandas df using read_csv
Convert the df to dictionary using to_dict

import pandas as pd
import io
import requests

link = "https://datasets/file.csv"
csv=requests.get(link).content
df=pd.read_csv(io.StringIO(csv.decode('utf-8')))
records = df.to_dict(orient='records'))
print (records)

Creating a list of dictionaries from a URL that points to a CSV

1 Answers1