9

I have a dict of arrays

{1:array([...]), 2:array([...]), 3:array([...])}

I'd like to save it to a file and load it back later.

I found a numpy had a list of Input and output methods, but seems they only deal with arrays.

Thanks.

Harrison
  • 2,560
  • 7
  • 29
  • 55

3 Answers3

13

The following script will save an dict of numpy arrays to disk and then load it back in to memory.

import numpy as np

arr1 = np.arange(0, 10, 1)
arr2 = np.arange(10, 20, 1)
arr3 = np.arange(20, 30, 1)

dct = {'1': arr1, '2': arr2, '3':arr3}

outfile = 'dict_of_arrays.npz'
np.savez(outfile, **dct)

npzfile = np.load(outfile)
print('npzfile.files: {}'.format(npzfile.files))
print('npzfile["1"]: {}'.format(npzfile["1"]))

Running this script shows the following:

npzfile.files: ['1', '3', '2']
npzfile["1"]: [0 1 2 3 4 5 6 7 8 9]

Note that your dict keys must be strings. Perhaps that was the problem?

I am running numpy 1.10.4

Eric Dill
  • 2,166
  • 2
  • 17
  • 15
  • Do I need to loop through the `npzfile` to get my `dct` back, or there is an easier way? Converting all key to strings is a bit inconvenient though. – Harrison Mar 02 '16 at 01:51
  • What kinds of `key` values are you using? In an `npz` each array is saved in a file whose name match the array's `name/key`. The `npz` itself is a `zip` archive of these files. – hpaulj Mar 02 '16 at 01:54
  • 1
    `npzfile` behaves like a dict and you can iterate over it. It does support item assignment, so you cannot do things like `npzfile[4] = array`, but you can just do `dct = dict(npzfile)` and then you have a dictionary that you can assign new elements to. – Eric Dill Mar 02 '16 at 01:55
  • Note that `npzfile` is a lazy loader; a specific array is extracted and loaded only when the `key` is accessed. – hpaulj Mar 02 '16 at 03:13
  • Oh, thanks for pointing that out. I was not aware that it was lazy, but it makes good sense that it is – Eric Dill Mar 02 '16 at 04:56
4

Actually you can use built-in pickle library to serialize and deserialize your objects without using numpy.

Here is a mock code

import pickle

data1 = {'a': [1, 2.0, 3, 4 + 6j],
         'b': ('string', u'Unicode string'),
         'c': None}

print data1, type(data1)
with open('data.pkl', 'wb') as output:
    # Pickle dictionary using protocol 0.
    pickle.dump(data1, output)

# load data from pkl file
with open("data.pkl", "rb") as fp:
    loaded_data1 = pickle.load(fp)

print loaded_data1, type(loaded_data1)

Results

Before: {'a': [1, 2.0, 3, (4+6j)], 'c': None, 'b': ('string', u'Unicode string')} <type 'dict'>
After: {'a': [1, 2.0, 3, (4+6j)], 'c': None, 'b': ('string', u'Unicode string')} <type 'dict'>

Hope it helps.

Eric
  • 2,636
  • 21
  • 25
1

Use scipy.io.savemat()/loadmat():

import scipy.io as sio
sio.savemat(filename, pydict)
...
pydict = sio.loadmat(filename)

This saves dictionary of names and arrays in Matlab-style .mat file.

Advantages:

1) You may save several arrays (including numpy ones) into a single file in binary format which uses much less HD space than saving into json (for example);

2) You may save and restore not only 1-D arrays but also multi dimensional ones.

Notice: may be you will have to use strings as keys instead of int.

Alexey Antonenko
  • 2,389
  • 1
  • 18
  • 18