3

I'm trying to save a numpy array to disk but the saving process never seems to end, and the terminal keeps hanging without showing me the final print message of 'done'. I can tell the array is correctly formed by checking its shape right before the saving function. I wonder if I'm doing anything wrong or if this would be expected with numpy.save given the size of the dataset.

Is there a more efficient way of doing this either with numpy or h5py?

# Xy_array is a 3D array of 10,000 images, each 224 x 224

Xy_array = np.asarray(lst)
f = open('./data/data.npy', 'wb')
np.save(f, Xy_array)
f.close()
print('done')
pepe
  • 9,799
  • 25
  • 110
  • 188
  • What's `Xy_array.dtype`? – Ilja Everilä Jun 18 '16 at 22:02
  • good point-- it's a `dtype('O')` -- does this mean I can't save a numpy object? – pepe Jun 18 '16 at 22:12
  • Isn't `dtype('O')` objects? Apparently [`numpy.save()`](http://docs.scipy.org/doc/numpy-1.10.0/reference/generated/numpy.save.html) can pickle, but that might produce massive amounts of data, as your array is already 595 million items, IIUC. – Ilja Everilä Jun 18 '16 at 22:20
  • 1
    maybe this anwser might help you http://stackoverflow.com/questions/30329726/fastest-save-and-load-options-for-a-numpy-array – spedy Jun 18 '16 at 22:22

0 Answers0