Save tuple of numpy arrays to disk?

Question

I'm getting a tuple of numpy arrays as (keypoint, descriptor) when I run the compute function to extract them from an image.

Is there a way to pack this tuple together so that I can save them to a file, or write them into a CSV as a row?

You can save your tuple to a file straight away, e.g. with ``pickle``. — fjarri, Mar 18 '15 at 01:20
`np.savez` can save several arrays to a ZIP archive. `np.savetxt` writes a CSV like text file. It just arranges its input into a 2d array, and then writes it (with a format string), one row per file line. Nothing fancy. — hpaulj, Mar 18 '15 at 01:34
I just noticed that my suggested edit to add a `python-2.x` tag was rejected, and I can't figure out why. — KSFT, Apr 11 '15 at 16:31

score 2 · Accepted Answer · edited May 23 '17 at 12:14

2

There are a few ways you can do this:

You can use Python csv module, specifically writer objects:
```
import csv
writer = csv.writer(open("file.csv","w"))
for row in array:
     writer.writerow(str(row))
```
According to this question, there might be some formatting problems with this.
As mentioned in the comments, you can use numpy.savetxt(). This is probably the best way:
```
numpy.savetxt("file.csv",array,delimiter=",")
```
You can also use the pickle module:
```
import pickle
pickle.dump(array,open("file","w"))
```
As mentioned in the docs I linked above, pickle.load() should not be used to load untrusted data.

edited May 23 '17 at 12:14

Community

answered Mar 18 '15 at 03:06

KSFT

Pickle seems to be the easiest way of storing a tuple of numpy arrays. Thanks! – user339946 Mar 18 '15 at 05:06
`csv.writer()` takes a **file object**, not a filename; `with open('file.csv', 'wb') as outf: writer = csv.writer(outf)`. – Martijn Pieters Apr 11 '15 at 16:46
@MartijnPieters I just noticed that when checking for differences in Python 2 vs. Python 3. I fixed it. – KSFT Apr 11 '15 at 16:51
i don't think you mean to be using the `.csv` file suffix in the `pickle` dump. also, shouldn't you be opening it in binary mode (i.e., `'wb'`)? – abcd Apr 11 '15 at 17:13
You're right about the first point (though the extension doesn't really matter, but `pickle.dump()` doesn't need a file opened in binary mode. – KSFT Apr 11 '15 at 17:15
ah, ok, good to know. i always use `'wb'`. the extension doesn't _really_ matter, but at the same time it does matter, because a non-CSV file with a `.csv` suffix can cause a lot of confusion. – abcd Apr 11 '15 at 17:17
why not give it a pickle file suffix, like `.p` or `.pkl`? – abcd Apr 11 '15 at 17:18
I just noticed that I left an open parenthesis without a closing one up there, and that comment is too old to edit.) – KSFT May 10 '15 at 05:28

abcd · Answer 2 · 2015-04-11T17:20:46.727

Rather than using pickle, I would recommend using cPickle:

import cPickle as pickle
pickle.dump(array, open('file.p', 'wb'))

Note that all of your calls remain the same as when you are using pickle -- only the import line changes.

cPickle is much faster than pickle and with large arrays can save you a lot of time.

That said, when you're dealing with numpy arrays, using numpy's save functions -- e.g., np.savetxt -- is your best bet.

2 Answers2