Python write and read list of arrays

Question

Hey I am new to python and am working on some project where I have a list of arrays like -

some_list = [array([-12, 23]), array([-13, 22])]

These array represent unique property of some stuff. So I want to save this in a file like Json with some Id assigned to each array and then extract it back as the same list of arrays.

Are these Numpy arrays, stdlib `array.array`s, or something different? — abarnert, May 30 '18 at 18:09
Also, what do you mean by "some Id assigned to each array"? Getting them back in the same order as the original list makes sense, but you don't need IDs for that, and it's hard to see what you're do with them—you just want to preserve the order when you store and load them, which is exactly what happens when you save a Python `list` as a JSON `array`. — abarnert, May 30 '18 at 18:10
These array represent property of some known objects. Now I want to save them in a file and when I get an unknown object I want to compare it with my stored list of arrays and If matched return its id or else append it to the list with a unique id. — , May 30 '18 at 18:13
Related (?) [Numpy array is not JSON serializable](https://stackoverflow.com/q/26646362/2823755).. there are others. — wwii, May 30 '18 at 18:13
I tried using `tolist()` but then I was not able to get the same format back. — , May 30 '18 at 18:15
@kodeturtle `tolist()` will give you something that you can save—but then you need to reverse that at load time, e.g., with `array(…)`. — abarnert, May 30 '18 at 18:17
I wrote an answer to explain it. But meanwhile, I don't understand what you mean with the "These array represent property of some known objects." You have them in a big list, not a dict or something else keyed with IDs. If you want to save and load that list, that's easy. If you want some completely different structure… first you have to work out what that structure is, and show us how you plan to use it, and then we can show you how to save and load it. — abarnert, May 30 '18 at 18:19
The array of list that I have are the properties of people. Each array represent a property of known person. I want to save it in a Json format with that person's name. Then when the need to identify a person arises I will generate the list of arrays and use it to find the name of the person which I already have stored in the file. — , May 30 '18 at 18:29
But if you could only help me with saving the list of arrays in a file and then recovering it face in the same format would be sufficient. — , May 30 '18 at 18:30

score 1 · Accepted Answer · answered May 30 '18 at 18:15

The json module only knows how to handle the basic Python types that it maps to the basic JSON types—list, dict, str, float, bool, and NoneType.

But you can override the JSON encoder and decoder objects to add in code to handle additional types in any way you want. This is documented, with some examples, but isn't entirely trivial.

But, more simply, if you know the structure of the data you're trying to save and load, you can just transform it on the fly. For example:

def save_stuff(f, list_of_arrays):
    list_of_lists = [list(arr) for arr in list_of_arrays]
    json.dump(f, list_of_lists)

def load_stuff(f):
    list_of_lists = json.load(f)
    list_of_arrays = [np.array(lst) for lst in list_of_lists]
    return list_of_arrays

Or you can go the opposite direction: convert the list of arrays to an array of arrays and use Numpy to save that.

Either way, this is less efficient, because you have to create those temporary lists. If you have gigantic arrays, or if you're doing this zillions of times, you will probably want to do the more complicated work of overriding the encoder and decoder.

Python write and read list of arrays

1 Answers1