Python multiprocessing with shared numpy array

Question

Suppose I created an object A with an 2 dimension numpy array as attributes. Then I created 10 threads using Process API to randomly set the rows of A.

I want to know if I write the following code, whether self.x if shared among all the Process(thread), or every Process(thread) has just a copy?

If not shared, I will lose all the updates, right?

import numpy as np
from multiprocessing import Process

class A:

   def __init__():
       self.x = np.zeros((3,4))

   def update():
        threads = []
        for i in range(10):
            trd = Process(target=self.set, args=(i,))
        threads.append(trd)
        trd.start()

        for i in range(10):
            threads[i].join()

   def set(i):
       self.x[i/3] = np.random.rand(1,4)


if ___main___:
        a = A()
        a.update()

Answer 1

No, it is not shared. You spawn multiple processes with each process copying the file descriptor of the parent process and with no shared object.

To create shared a shared variable you have use ctype objects.

So instead of declaring the array as -

self.x = np.zeros((3,4))

you can declare it using this Array -

from multiprocessing import Array
self.x = Array('i', [0]*10)

If still you want to make the numpy array a shared array, have a look at this great answer.

The caveat here is, it might not be that easy. You'll also have to lock the shared array to avoid any race condition.