How to append arrays to an existing .npy file

Question

The goal of this program is to append a small data array of size [1,3] for each function call to an existing .npy file. This .npy file will be used for subsequent analysis in some other program. What's important is that it's populated with all the data arrays that were collected.

So here's the general overview of what I'm trying to accomplish (all within a single class):

1. Load an existing .npy file as an ndarray that may or may not be empty

2. Create a function that appends an array to this existing file whenever called

I've ran into several issues trying to accomplish this: Firstly, using np.load() doesn't work for an empty array, regardless of whether allow_pickles=True or allow_pickles=False. Secondly, every time I try to make a function that achieves this, the data is never appended, but rather overwritten as far as I can tell.

I have used different combinations of np.load() and np.save() for ndarrays. I've also tried append() for lists and then converting them into ndarrays. Here's some sample code that I've tested in a Jupyter Notebook:

import numpy as np

def appendArrayToFile(filename, array):
    np.save(filename,array)
    np.load(filename)
    return

appendArrayToFile('probedZdata.npy', np.array([1,2,3]))
appendArrayToFile('probedZdata.npy', np.array([4,5,6]))

print(np.load('probedZdata.npy'))

Instead of returning all of the arrays, it only returns the last one:

[4, 5, 6]

How can I format this function such that I add an array to this file each time it's called? The size of this file is mostly irrelevant as I would only be storing up to around 50, 3x1 sized arrays.

Answer 1

You are overwriting the file. You're not adding.