write and read lists from/to file in python

Question

How to write, then read, (conserving all specifics) of the following list, in python?

Using various methods, I could not read the data back with the exact same formatting, datatypes, etc. I'm using python 3.6.7. Here is a toy-sample to play with

sample_list = [[np.ones(shape = (3,4), dtype='uint8'), np.int64(2), 'd654'], [np.ones(shape = (3,4), dtype='uint8'), np.int64(4), 'd654']]

No. That seems to solve the problem. – Kevin Preston Dec 27 '19 at 18:09 — Kevin Preston, Dec 27 '19 at 18:09

Ekaba Bisong · Accepted Answer · 2019-12-30T15:23:09.723

1

Import Pickle:

import pickle

Save Variable:

f = open('store.pckl', 'wb')
pickle.dump(sample_list, f)
f.close()

Load Variable:

f = open('store.pckl', 'rb')
obj = pickle.load(f)
f.close()

Reference: https://stackoverflow.com/a/6568495/3353760

Update: Using numpy.save()

import numpy as np

Save Variable:

np.save(file='sample_list', arr=sample_list)

Load Variable:

np.load('sample_list.npy', allow_pickle=True)

edited Dec 30 '19 at 15:23

answered Dec 27 '19 at 18:06

Ekaba Bisong

2,918
2
23
38

Unfortunately, pickled failed to save to file due to "MemoryError: ". I postulate that the data to be save was too big so I used: np.save('X',X) np.load('X.npy') – Kevin Preston Dec 28 '19 at 19:20
Right. 'np.save()' works better with saving large matrices and loading them back into memory. – Ekaba Bisong Dec 28 '19 at 23:07
I'm not on my computer now, feel free to update the answer. Else I'll do it later... – Ekaba Bisong Dec 28 '19 at 23:08

write and read lists from/to file in python

1 Answers1