I have numpy arrays around 5 gig and working on python 3.x ubuntu 18.
When I try to numpy.save its consume nearly 2,5X memory. Is there any way to dump this data to disk without consume so much memory ?
Best
Asked
Active
Viewed 93 times
0
-
Possibly [related](https://stackoverflow.com/questions/47857782/run-out-of-memory-when-saving-list-with-numpy). – hilberts_drinking_problem Aug 08 '19 at 06:30
-
If you're saving big numeric data to disk, you might want to consider using HDF5, which is designed for that. See [this thread](https://stackoverflow.com/questions/9619199/best-way-to-preserve-numpy-arrays-on-disk?rq=1). – eaglesear Aug 08 '19 at 09:32