How can I save a list of dictionaries to a file?

Question

I have a list of dictionaries. Occasionally, I want to change and save one of these dictionaries so that the new message is utilized if the script is restarted. Right now, I make that change by modifying the script and rerunning it. I'd like to pull this out of the script and put the list of dictionaries into a configuration file of some kind.

I've found answers on how to write a list to a file, but this assumes that it is a flat list. How can I do it with a list of dictionaries?

My list looks like this:

logic_steps = [
    {
        'pattern': "asdfghjkl",
        'message': "This is not possible"
    },
    {
        'pattern': "anotherpatterntomatch",
        'message': "The parameter provided application is invalid"
    },
    {
        'pattern': "athirdpatterntomatch",
        'message': "Expected value for debugging"
    },
]

what do you think of jsonl/json lines: https://ml-gis-service.com/index.php/2022/04/27/toolbox-python-list-of-dicts-to-jsonl-json-lines/? — Charlie Parker, Aug 10 '22 at 20:44

score 76 · Accepted Answer · answered Sep 11 '14 at 02:20

76

provided that the object only contains objects that JSON can handle (lists, tuples, strings, dicts, numbers, None, True and False), you can dump it as json.dump:

import json
with open('outputfile', 'w') as fout:
    json.dump(your_list_of_dict, fout)

answered Sep 11 '14 at 02:20

mgilson

300,191
65
633
696

6

I get the error TypeError: -0.69429028 is not JSON serializable – Mona Jalal Apr 21 '18 at 21:21
As I was importing my data from the file I needed to add `fout.write("data = ")` before the `json.dump` – Cyrus Jun 18 '21 at 05:11
what about jsonl/json lines? – Charlie Parker Aug 10 '22 at 20:44

score 19 · Answer 2 · answered Apr 30 '18 at 19:43

19

if you want each dictionary in one line:

 import json
 output_file = open(dest_file, 'w', encoding='utf-8')
 for dic in dic_list:
    json.dump(dic, output_file) 
    output_file.write("\n")

answered Apr 30 '18 at 19:43

Reihan_amn

2,645
2
21
21

what is dest_file? – Eliza R Dec 12 '20 at 06:12
@ElizaR dest_file is the location of the destination file. This can be something like /home/user/file on linux – zwep Jul 16 '21 at 13:36
what about jsonl/json lines? – Charlie Parker Aug 10 '22 at 20:44

Nikos Tavoularis · Answer 3 · 2018-02-07T13:01:30.297

7

Just for completeness I add also the json.dumps() method:

with open('outputfile_2', 'w') as file:
    file.write(json.dumps(logic_steps, indent=4))

Have a look here for the difference between json.dump() and json.dumps()

edited Feb 07 '18 at 13:01

answered Dec 12 '17 at 00:03

Nikos Tavoularis

2,843
1
30
27

score 3 · Answer 4 · answered Sep 11 '14 at 02:56

The way you will have to follow to write a dict to a file is kind different from the post you have mentioned.

First, you need serialize the object and than you persist it. These are fancy names for "write python objects to a file".

Python has 3 serialization modules included by default that you can use to achieve your objective. They are: pickle, shelve and json. Each one has its own characteristics and the one you have to use is the one which is more suitable to your project. You should check each module documentation to get more on it.

If your data will be only be accessed by python code, you can use shelve, here is an example:

import shelve

my_dict = {"foo":"bar"}

# file to be used
shelf = shelve.open("filename.shlf")

# serializing
shelf["my_dict"] = my_dict

shelf.close() # you must close the shelve file!!!

To retrieve the data you can do:

import shelve

shelf = shelve.open("filename.shlf") # the same filename that you used before, please
my_dict = shelf["my_dict"]
shelf.close()

See that you can treat the shelve object almost the same way you do with a dict.

score 0 · Answer 5 · answered Mar 07 '23 at 18:09

The accepted answer to this question works flawlessly for this problem as well. Instead of json, it makes use of pickle to manage data dumping transparently, even in the case of a list of dictionaries:

import pickle

list_of_dicts = [{'a': 1, 'b': 2}, {'aa': 2, 'bb': 3}]

with open('saved_list.pkl', 'wb') as f:
    pickle.dump(list_of_dicts, f)
        
with open('saved_list.pkl', 'rb') as f:
    loaded_list_of_dicts = pickle.load(f)

How can I save a list of dictionaries to a file?

5 Answers5

Linked