Can't save variable when using multiprocessing

Question

from multiprocessing import Process


class B:
    def __init__(self, a, b):
        self.a = a
        self.b = b


class A:
    def __init__(self, a):
        self.k = None
        self.k2 = None
        p1 = Process(target=self.a1, args=(a, 1,), daemon=True)
        p2 = Process(target=self.a2, args=(a, 2,), daemon=True)

        p1.start()
        p2.start()

        p1.join()
        p2.join()

    def a1(self, a, b):
        self.k = B(a, b)

    def a2(self, a, b):
        self.k2 = B(a, b)


if __name__ == '__main__':
    t = A(0)
    print(t.k)

The project plans that class B will be large and will contain a large number of calculations. I wanted to divide the creation of a class into streams, but I can’t save the variable k. Why is the variable k always None? How to fix?

Does this answer your question? [Python Treading](https://stackoverflow.com/questions/64613951/python-treading) — Robert Bradley, May 11 '23 at 15:14
Because you are not creating threads but fork entirely separate processes. These new processes can not write back into the parent process directly. You have to use cross-process data strictures such as Pipes or shared memory. See https://docs.python.org/3/library/multiprocessing.html#sharing-state-between-processes — Holger Just, May 11 '23 at 15:17

score 0 · Answer 1 · answered May 12 '23 at 19:10

As commented by Holger Just, the child processes are being passed a copy of the A instance (in variable t) and so when self.k and self.k2 are set to new values, it is not being reflected back to the copy of the A instance this is within the main process (i.e. t).

There are several possible solutions to this problem but since what you are trying to assign to these properties are B instances rather than a built-in type (such as an int), the easiest solution I think is to use a NameSpace managed object as the internal state of A and to which arbitrary properites can be assigned. This is a large subject that you need to read up on, but the modified code would look like:

from multiprocessing import Process, Manager


class B:
    def __init__(self, a, b):
        self.a = a
        self.b = b

    def __repr__(self):
        return f'B({self.a}, {self.b})'


class A:
    def __init__(self, manager, a):
        self.state = manager.Namespace()
        self.state.k = None
        self.state.k2 = None
        p1 = Process(target=self.a1, args=(a, 1,), daemon=True)
        p2 = Process(target=self.a2, args=(a, 2,), daemon=True)

        p1.start()
        p2.start()

        p1.join()
        p2.join()

    def a1(self, a, b):
        self.state.k = B(a, b)

    def a2(self, a, b):
        self.state.k2 = B(a, b)


if __name__ == '__main__':
    with Manager() as manager:
        t = A(manager, 0)
        print(t.state.k)
        print(t.state.k2)

Prints:

B(0, 1)
B(0, 2)

Can't save variable when using multiprocessing

1 Answers1