Using pickle in python to store dictionary

Question

I wrote this code that calculate the hash value for a pdf file and add it to a dictionary and save it to a file like this:

v={hash_value:{"file name":file_name,"number":1}}

But the problem with the below code is that if I add a new file it overwrite the previous one.

f = calculate_hash("calc.pdf")
v = {f:{"file name":"calc.pdf","number":1}}

with open('filename.pickle', 'wb') as handle:
    pickle.dump(v, handle)  

with open('filename.pickle', 'rb') as handle:
    b = pickle.load(handle)

x=  calculate_hash("calc.pdf")
for key in b:
    print key
    if x == key:
        print "yes"

Yes, `open('filename.pickle', 'wb')` clears the file - is the case that you wanted first to load the distionary(s), amend them, and then store again? — mdurant, Jul 30 '14 at 13:35
The why don't you read the file first and write it back after the modifications? — Matthias, Jul 30 '14 at 13:50

score 1 · Accepted Answer · answered Jul 30 '14 at 13:38

1

Just use 'append' mode:

with open('filename.pickle', 'wb') as handle:

=>

with open('filename.pickle', 'ab') as handle:

answered Jul 30 '14 at 13:38

Evgeny Prokurat

704
5
8

3

Although generally a correct way to add to the end of a file, this is more complex with `pickle`. You will then need to deal with the multiple objects separately on `load`: see e.g. http://stackoverflow.com/a/15463472/3001761 – jonrsharpe Jul 30 '14 at 13:39

score 0 · Answer 2 · answered Jul 30 '14 at 13:43

0

when you open a file, you need to use the 'a' for append, instead of 'w'.

see reading-and-writing-files

answered Jul 30 '14 at 13:43

user1438233

1,153
1
14
30

When adding a link to your answer you should include an extract of the relevant information provided by the link. – Mizipzor Jul 30 '14 at 15:11

score 0 · Answer 3 · answered Jul 30 '14 at 15:20

The main error here is that you write to the file before you read it. Thereby overwriting all existing values rather than combining the existing values with the new one.

How about this?

import pickle
import random

def calculate_hash(s):
    return random.randint(1, 100) # not proper hash

f = calculate_hash("calc.pdf")
v = {f:{"file name":"calc.pdf","number":1}}

# read before you write
with open('filename.pickle', 'rb') as handle: # file has to exist here
    b = pickle.load(handle)

# combine the two dictionaries
b = dict(b.items() + v.items())

# write the combined dictionaries
with open('filename.pickle', 'wb') as handle:
    pickle.dump(b, handle)  

x=  calculate_hash("calc.pdf")
for key in b:
    print key
    if x == key:
        print "yes"

Using pickle in python to store dictionary

3 Answers3