In a multiprocess situation, I want to avoid reading arrays from a zarr group that haven't fully finished writing by the other process yet. This functionality does not seem to come out of the box with zarr.
While chunk writing is atomic in zarr, array writing seems not to be (i.e. while you can never have a half-written chunk, you can have a half-written array if said array spans multiple chunks).
In my concrete example, one process is writing to the position group. This group contains a 1D array with a chunksize of 100. All goes well if the array I'm writing is smaller than this chunksize. Larger arrays will be written into several chunks, but not all of them are written simultaneously.
A parallel process may then try to read the array and find only a first chunk. Zarr then blithely returns an array of 100 elements. Milliseconds later, the 2nd chunk is written, and a subsequent opening of the group now yields 200 elements.
I can identify a number of solutions:
A store/group lock which must be acquired before writing or reading the entire array. This works, but makes concurrent writing and reading a lot harder because chunk-level locking is better than group/store-level locking. For simple 1D arrays that are write once/read many, that's enough.
A store/group lock that does not allow reading the entire array while the array is write-locked. I don't know if such read/write locks exist in zarr, or if I should brew my own using the fasteners library. Again for more complex N-D arrays this means loss of performance.
Adjust my write/read code to obtain a lock based on the region to write or read (the lock key could be composed of the indices to write or chunks to write). This would have better performance but it seems absurd that this isn't out-of-the-box supported by zarr.
The zarr docs are a bit too succinct and don't delve very deep into the concept of synchronisation/locking, so maybe I'm just missing something.
