I am currently working on training a neural network and the preprocessing step takes a while to go through, especially when I want to tweak the parameter of my network structure. My input data consist of a list of numpy arrays, stored in 4 different variables. Is possible to store these list of numpys into local files, such that I just could load them and start the training rather than loading the raw files and begin preprocess that.
Asked
Active
Viewed 1,676 times
0
-
Read the docs of ```np.savez()```. – sascha Mar 12 '17 at 15:55
-
Yes. I just saw that.. can i give a list a input..? – user7654132 Mar 12 '17 at 15:57
-
Give it the output filename/path first, then each object individually. – Ari Cooper-Davis Mar 12 '17 at 16:01
-
@AriCooper-Davis I am not sure I understand? – user7654132 Mar 12 '17 at 16:06
-
1@user7654132 Check [this SO-question about variable arguments](http://stackoverflow.com/questions/919680/can-a-variable-number-of-arguments-be-passed-to-a-function) and the [official docs](https://docs.python.org/3/tutorial/controlflow.html#arbitrary-argument-lists). – sascha Mar 12 '17 at 16:19
-
1@user7654132 see my edited answer for a working example. – Ari Cooper-Davis Mar 12 '17 at 16:22
1 Answers
2
You can save numpy arrays using numpy.save (or numpy.savez for multiple arrays at once) and load them again using numpy.load.
e.g. saving arrays:
import numpy
test_array_1 = numpy.ones([5, 6]) #creating a test array
test_array_2 = numpy.zeros([6, 7]) #creating a test array
numpy.savez("testfile.npz", test_array_1=test_array_1, test_array_2=test_array_2) #saving my two test arrays
e.g. loading arrays
import numpy
data = numpy.load("testfile.npz") #loading the two arrays into a variable called 'data'
print(data["test_array_1"]) #using one array directly
test_array_2 = data["test_array_2"] #assigning one to a varibale
Ari Cooper-Davis
- 3,374
- 3
- 26
- 43
-
-
If you're always saving and loading those 4 together then yes, savez would be fine. – Ari Cooper-Davis Mar 12 '17 at 15:58