How to assign pickle name dynamically in Python?

Question

I have a loop, and I'd like to pickle all outputs. The problem is that I can't assign my variables as a pickle name. I might make it clear with this little example:

lst = [a,d,e,g,h,j]
for i in lst:
    pickle.dump(i,open('xx.p','wb'))

As I said before, I would like to assign different names for each i in my list. In other words, I want to assign my variables values as a pickle name. ('a.p' -> for first, 'd.p' -> for second and so on).

How about using a dictionary instead of a list? `{'a': a, 'b': b}` — Chris Eberle, Feb 17 '13 at 16:57

score 3 · Accepted Answer · answered Feb 17 '13 at 17:48

You can do this easily if you use two lists, one for your objects, and one for what you want to name them:

lst = [a,d,e,g,h,j]
nm_lst = ['a', 'b', 'd', 'e', 'h', 'j']
for i,nm in zip(lst,nm_lst):
    pickle.dump(i,open('{0}.p'.format(nm),'wb'))

Or you can do this with a dictionary (as suggested in the comments)

my_dict = {'a':a, 'b':b}
for key, val in my_dict.iteritems():
    pickle.dump(val,open('{0}.p'.format(key),'wb'))

(if you are using python3, iteritems -> items)

piokuc · Answer 2 · 2013-02-17T18:13:08.467

You can find variable's name by examining environment where it's defined. Here you have two functions exploring this idea, one for pickling, one for unpickling.

You need to pass the environment to the pickling function in order to find out variables names, create a dictionary mapping the names to variables and, eventually, pickle the dictionary containing variables and their names.

The unpickling function will update the environment (usually locals()) you pass to it with unpickled variables from your file.

import pickle

def pickle_vars(fileName, env, *vs):
    d = dict([(x, env[x]) for v in vs for x in env if v is env[x]])
    with open(fileName, 'wb') as f:
        pickle.dump(d, f)

def unpickle_vars(fileName, env):
    with open(fileName, 'rb') as f:
        d = pickle.load(f)
        env.update(d)


def f():
    x = 10
    y = 20
    pickle_vars('vars', locals(), x, y)

f() 
unpickle_vars('vars', globals())
print x,y

Here is a modified function to pickle each variable in separate file, named as the variable with extension .p:

def pickle_vars(fileName, env, *vs):
    d = [(x, env[x]) for v in vs for x in env if v is env[x]]
    for name, var in d:
        with open(name + '.p', 'wb') as f:
            pickle.dump(var, f)

The op wants to put each variable in a separate file. – tacaswell Feb 17 '13 at 18:04 — tacaswell, Feb 17 '13 at 18:04
OK, I added a function for doing exactly that. – piokuc Feb 17 '13 at 18:13 — piokuc, Feb 17 '13 at 18:13

How to assign pickle name dynamically in Python?

2 Answers2