Python3 from a list (dict on every line) to csv

Question

I have a list with a dictionary type every line, so basically I can walk thru the list in a loop to get each dictionary record. I need to create a csv file from that list and every record is each of the items dictionary on the list.

My list look like :

[{"field1":"field","field2":"field"....},{"field1":"field","field2":"field"....}]

I have done this snippet code below but I'm storing only the headers:

with open('myfile', mode='wt', encoding='utf-8') as myfile:
    for lines in data:
        myfile.write('\n'.join(lines))
        myfile.write('\n')

I was exploring How do I write a Python dictionary to a csv file? but I dont know how to compose this to each line of my list within a loop to guarantee every record is coming from the line list.

I appreciate any help

thanks.

can you be more specific on what `data` is? If that is a dictionary, perhaps you should use `lines.values()` instead of `lines` inside the `join()`. — norok2, Sep 25 '18 at 16:20

score 1 · Answer 1 · answered Sep 25 '18 at 16:26

with open('myfile', 'wb') as f:
    # assign your headers
    w = csv.DictWriter(f, fieldnames=data[0].keys())
    # write headers
    w.writeheader()
    # write data
    w.writerows(data)

You can use DictWriter.writerows to iterate over a list of data.

You can see in the docs how DictWriter works.

And here is the spec for writerows:

csvwriter.writerows(rows)¶

Write all elements in rows (an iterable of row objects as described above) to the writer’s file object, formatted according to the current dialect.

Paulo Scardine · Answer 2 · 2018-09-25T16:28:04.647

It is almost a duplicate of the other question but the comment system is not suitable for a code snippet this big. Just use a for loop iterating over your list of dicts where the answer for the other question is persisting only one row:

with open('myfile', mode='wt', encoding='utf-8') as myfile:
    writer = csv.DictWriter(myfile, data[0].keys())
    writer.writeheader()
    for line in data:
        writer.writerow(line)

You can also use the writerows method and pass the whole list:

with open('myfile', mode='wt', encoding='utf-8') as myfile:
    writer = csv.DictWriter(myfile, data[0].keys())
    writer.writeheader()
    writer.writerows(data)

Joe Patten · Accepted Answer · 2018-09-25T16:31:06.157

You can actually use pandas to solve this quite nicely:

import pandas as pd
df = pd.DataFrame(dict_list)
df.to_csv('output.csv', index=False)

Here's a demo:

dict_list = [{'field1': 100, 'field2': '10:00', 'field3': 'cherries'}, 
{'field1': 500, 'field2': '6:00', 'field3': "cheese"}, 
{'field1':250, 'field2': '9:00', 'field3': 'beans'}]

Output

   field1 field2    field3
      100  10:00  cherries
      500   6:00    cheese
      250   9:00     beans

Python3 from a list (dict on every line) to csv

3 Answers3