Python Convert CSV to Dictionary of Lists

Question

I have a CSV file that looks like this:

compound, x1data,y1data,x2data,y2data
a,1,2,3,4
a,9,10,11,12
b,5,6,7,8
b,4,5,6,7

I would like to create a dictionary of lists where the compound is the key and for each compound I get a list of x1data, y1data, x2data, and y2data.

I believe it would look something like this:

my_dict = {
    'a': {'x1data':[1,9],'y1data':[2,10],'x2data':[3,11],'y2data':[4,12]},
    'b':{'x1data':[5,4],'y1data':[6,5],'x2data':[7,6],'y2data':[8,7]}
}

Ultimately I want to plot x1data vs y1data and x2data vs y2data for each of the compounds.

I've tried this which correctly makes a dictionary where the keys are compounds, but it doesn't give me a list of values (just the last value in the csv.

my_dict = {}
with open(filename, 'r') as infile:
    reader = csv.DictReader(infile)
    for row in reader:
        key = row.pop('compound')
        my_dict[key] = row

score 0 · Answer 1 · answered Aug 30 '19 at 17:11

here's a way to do it without any libraries.

f = open('f.csv', 'rb')
next(f)
mydict = {}
for row in f:

    compound,x1data,y1data,x2data,y2data = row.strip().split(',')
    x1data,y1data,x2data,y2data = int(x1data),int(y1data),int(x2data),int(y2data)
    if compound not in mydict:
        mydict[compound] = { 'x1data' : [], 'y1data' : [], 'x2data' : [], 'y2data' : [] }


    mydict[compound]['x1data'].append(x1data)
    mydict[compound]['y1data'].append(y1data)
    mydict[compound]['x2data'].append(x2data)
    mydict[compound]['y2data'].append(y2data)
}
f.close()

print mydict

gives you:

{'a': {'x2data': [3, 11], 'y2data': [4, 12], 'y1data': [2, 10], 'x1data': [1, 9]}, 'b': {'x2data': [7, 6], 'y2data': [8, 7], 'y1data': [6, 5], 'x1data': [5, 4]}}

score 0 · Answer 2 · answered Aug 30 '19 at 17:19

You can use collections.defaultdict from the standard library.

from collections import defaultdict as dd

import csv

my_dict = dd(lambda: dd(list))

with open("test.csv", 'r') as f:
    reader = csv.DictReader(f)

    for row in reader:
        for key in reader.fieldnames[1:]:
            my_dict[row.get("compound")][key].append(row[key])

Technically, what you get here isn't a dict. You can use it in the same way, though.

If you want to print, it's a bit more involved:

from pprint import pprint

# ...

pprint({k: dict(v) for k, v in dict(my_dict).items()})

This gives:

{'a': {'x1data': ['1', '9'],
       'x2data': ['3', '11'],
       'y1data': ['2', '10'],
       'y2data': ['4', '12']},
 'b': {'x1data': ['5', '4'],
       'x2data': ['7', '6'],
       'y1data': ['6', '5'],
       'y2data': ['8', '7']}}

Thanks! This worked well for me and I appreciate that it is flexible for different data headers, number of columns and keys, etc. — Kyle, Aug 30 '19 at 18:42

Collin Phillips · Answer 3 · 2019-08-30T17:43:01.867

Here is a solution that doesn't rely on csv libraries and should work with a header of arbitrary size.

with open("dat.csv", 'r') as f:
    lines = f.read().splitlines()

headers = lines.pop(0).split(",")[1:] # names of the columns
results = {}
for line in lines:
  line = line.split(",")
  if line[0] not in results:
    results[line[0]] = {header:[] for header in headers}
  for i, header in enumerate(headers):
    results[line[0]][header].append(line[i+1])
    # for ints: results[line[0]][header].append(int(line[i+1]))

print(results)

Output:

{'a': {'x2data': ['3', '11'], 'y2data': ['4', '12'], 'y1data': ['2', '10'], 'x1data': ['1', '9']}, 'b': {'x2data': ['7', '6'], 'y2data': ['8', '7'], 'y1data': ['6', '5'], 'x1data': ['5', '4']}}

The only change I made was removing a space in the provided header (it would work work either way though).

score 0 · Answer 4 · answered Aug 30 '19 at 17:38

You can use itertools.groupby:

import csv, itertools
[_, *hs], *data = csv.reader(open('filename.csv'))
r = [(a, [list(map(int, i[1:])) for i in b]) for a, b in itertools.groupby(data, key=lambda x:x[0])]
final_result = {a:dict(zip(hs, map(list, zip(*b)))) for a, b in r}

Output:

{'a': {'x1data': [1, 9], 'y1data': [2, 10], 'x2data': [3, 11], 'y2data': [4, 12]}, 'b': {'x1data': [5, 4], 'y1data': [6, 5], 'x2data': [7, 6], 'y2data': [8, 7]}}

Python Convert CSV to Dictionary of Lists

4 Answers4