Save a dictionary to a file (alternative to pickle) in Python?

Question

Answered I ended up going with pickle at the end anyway

Ok so with some advice on another question I asked I was told to use pickle to save a dictionary to a file.

The dictionary that I was trying to save to the file was

members = {'Starspy' : 'SHSN4N', 'Test' : 'Test1'}

When pickle saved it to the file... this was the format

(dp0
S'Test'
p1
S'Test1'
p2
sS'Test2'
p3
S'Test2'
p4
sS'Starspy'
p5
S'SHSN4N'
p6
s.

Can you please give me an alternative way to save the string to the file?

This is the format that I would like it to save in

members = {'Starspy' : 'SHSN4N', 'Test' : 'Test1'}

Complete Code:

import sys
import shutil
import os
import pickle

tmp = os.path.isfile("members-tmp.pkl")
if tmp == True:
    os.remove("members-tmp.pkl")
shutil.copyfile("members.pkl", "members-tmp.pkl")

pkl_file = open('members-tmp.pkl', 'rb')
members = pickle.load(pkl_file)
pkl_file.close()

def show_menu():
    os.system("clear")
    print "\n","*" * 12, "MENU", "*" * 12
    print "1. List members"
    print "2. Add member"
    print "3. Delete member"
    print "99. Save"
    print "0. Abort"
    print "*" * 28, "\n"
    return input("Please make a selection: ")

def show_members(members):
    os.system("clear")
    print "\nNames", "     ", "Code"
    for keys in members.keys():
        print keys, " - ", members[keys]

def add_member(members):
    os.system("clear")
    name = raw_input("Please enter name: ")
    code = raw_input("Please enter code: ")
    members[name] = code
    output = open('members-tmp.pkl', 'wb')
    pickle.dump(members, output)
    output.close()
    return members


#with open("foo.txt", "a") as f:
#     f.write("new line\n")

running = 1

while running:
    selection = show_menu()
    if selection == 1:
        show_members(members)
        print "\n> " ,raw_input("Press enter to continue")
    elif selection == 2:
        members == add_member(members)
        print members
        print "\n> " ,raw_input("Press enter to continue")
    elif selection == 99:
        os.system("clear")
        shutil.copyfile("members-tmp.pkl", "members.pkl")
        print "Save Completed"
        print "\n> " ,raw_input("Press enter to continue")

    elif selection == 0:
        os.remove("members-tmp.pkl")
        sys.exit("Program Aborted")
    else:
        os.system("clear")
        print "That is not a valid option!"
        print "\n> " ,raw_input("Press enter to continue")

I would like it to save as a regular dictionary E.g. members = {'Starspy' : 'SHSN4N', 'Test' : 'Test1'} — wakey, Feb 04 '11 at 02:20
See [Storing Python dictionaries](http://stackoverflow.com/q/7100125/562769) — Martin Thoma, Apr 29 '16 at 09:06

score 60 · Answer 1 · answered Feb 04 '11 at 01:34

60

Sure, save it as CSV:

import csv
w = csv.writer(open("output.csv", "w"))
for key, val in dict.items():
    w.writerow([key, val])

Then reading it would be:

import csv
dict = {}
for key, val in csv.reader(open("input.csv")):
    dict[key] = val

Another alternative would be json (json for version 2.6+, or install simplejson for 2.5 and below):

>>> import json
>>> dict = {"hello": "world"}
>>> json.dumps(dict)
'{"hello": "world"}'

answered Feb 04 '11 at 01:34

David Wolever

148,955
89
346
502

Thanks, can you still import .csv files? – wakey Feb 04 '11 at 01:38
2

CSV is a very ugly suggestion. It's used for storing tables of data, usually as exports from spreadsheets; it's not a format used for serializing data structures. – Glenn Maynard Feb 04 '11 at 01:56
4

This is very true. But reading between the lines, the OP is looking to store tuples of strings in a human-friendly format… And CVS is pretty decent for that. – David Wolever Feb 04 '11 at 02:04
Actually I ran into a snag, the file that loads has an initial value of "members = {'Starspy' : 'SHSN4N', 'Test' : 'Test1'}" but after I try to add the member "Amere : MT1PN" it saves in the file as "Test1,"{'Test1': 'Test', ""members = {'Starspy' : 'SHSN4N'"": "" 'Test' : 'Test1'}""}". The result I am trying to attain is where the new dictionary saves over the old one (erases the old one) and where it can be loaded up in the program again. I am fairly new at python so I hope im making sense. Ill add the complete code for my python program as it is right now into the OP. – wakey Feb 04 '11 at 02:19
I got it to work, I just went with Pickle eventually and after some well termed google searches I was able to find out what I needed to do. Thanks for your help! – wakey Feb 04 '11 at 03:12

score 56 · Accepted Answer · answered Feb 04 '11 at 01:40

56

The most common serialization format for this nowadays is JSON, which is universally supported and represents simple data structures like dictionaries very clearly.

>>> members = {'Starspy' : 'SHSN4N', 'Test' : 'Test1'}
>>> json.dumps(members)
'{"Test": "Test1", "Starspy": "SHSN4N"}'
>>> json.loads(json.dumps(members))
{u'Test': u'Test1', u'Starspy': u'SHSN4N'}

answered Feb 04 '11 at 01:40

Glenn Maynard

55,829
10
121
131

3

Ill give it a shot. How do I specify what file to dump it to/load it from? – wakey Feb 04 '11 at 02:25
10

Dump returns a string. You can just write this to any file as per normal python IO `with open('file.json', 'w') as f: f.write(json.dumps(members))` – Matt Clarkson Dec 02 '12 at 00:22
@AndreasAndreou There isn't one. The `u'Test'` format is Python, not JSON. – Glenn Maynard Mar 31 '14 at 16:32
@watsonic No, that's a Python feature. It's meaningless and illegal in JSON. – Glenn Maynard Jun 07 '14 at 00:38
@GlennMaynard, try this: `type(u'foo')`. what `json.loads()` returns is a dictionary typed in unicode. – watsonic Jun 07 '14 at 18:36
@watsonic That's the Python API. It has nothing to do with JSON. – Glenn Maynard Jun 07 '14 at 19:19
@MattClarkson: there is also `json.dump()` which writes directly to file. See http://stackoverflow.com/a/33569056/1587329. – serv-inc Nov 06 '15 at 14:31
1

I asked this question almost 10 years ago, and have to say this is actually the correct answer. – wakey Nov 05 '20 at 07:15

score 7 · Answer 3 · edited Jan 12 '17 at 21:35

7

Although, unlike pp.pprint(the_dict), this won't be as pretty, will be run together, str() at least makes a dictionary savable in a simple way for quick tasks:

f.write( str( the_dict ) )

edited Jan 12 '17 at 21:35

Tonechas

13,398
16
46
80

answered Feb 01 '14 at 04:26

gseattle

986
1
14
23

Since Python 3.2, this saved string object can be imported by [`ast.literal_eval`](https://docs.python.org/3/library/ast.html#ast.literal_eval) – Chris Aug 28 '17 at 09:17

score 7 · Answer 4 · answered Feb 04 '11 at 01:37

7

The YAML format (via pyyaml) might be a good option for you:

http://en.wikipedia.org/wiki/Yaml

http://pypi.python.org/pypi/PyYAML

answered Feb 04 '11 at 01:37

Pedro Matiello

99
2

1

That's a great idea. The only problem is that it requires an external module. – David Wolever Feb 04 '11 at 01:39
1

Yep. The external dependency is a nuisance, but YAML is cute. :) – Pedro Matiello Feb 04 '11 at 01:42

score 3 · Answer 5 · edited May 23 '17 at 11:54

3

You asked

Ill give it a shot. How do I specify what file to dump it to/load it from?

Apart from writing to a string, the json module provides a dump()-method, which writes to a file:

>>> a = {'hello': 'world'}
>>> import json
>>> json.dump(a, file('filename.txt', 'w'))
>>> b = json.load(file('filename.txt'))
>>> b
{u'hello': u'world'}

There is a load() method for reading, too.

edited May 23 '17 at 11:54

Community

1
1

answered Nov 06 '15 at 14:29

serv-inc

35,772
9
166
188

Mike McKerns · Answer 6 · 2014-03-06T04:28:11.640

While I'd suggest pickle, if you want an alternative, you can use klepto.

>>> init = {'y': 2, 'x': 1, 'z': 3}
>>> import klepto
>>> cache = klepto.archives.file_archive('memo', init, serialized=False)
>>> cache        
{'y': 2, 'x': 1, 'z': 3}
>>>
>>> # dump dictionary to the file 'memo.py'
>>> cache.dump() 
>>> 
>>> # import from 'memo.py'
>>> from memo import memo
>>> print memo
{'y': 2, 'x': 1, 'z': 3}

With klepto, if you had used serialized=True, the dictionary would have been written to memo.pkl as a pickled dictionary instead of with clear text.

You can get klepto here: https://github.com/uqfoundation/klepto

dill is probably a better choice for pickling then pickle itself, as dill can serialize almost anything in python. klepto also can use dill.

You can get dill here: https://github.com/uqfoundation/dill

Save a dictionary to a file (alternative to pickle) in Python?

6 Answers6

Linked