1

I know that child processes won't see changes made after a fork/spawn, and Windows processes don't inherit globals not using shared memory. But what I have is a situation where the children can't see changes to a global variable in shared memory made before the fork/spawn.

Simple demonstration:

from multiprocessing import Process, Value
global foo
foo = Value('i',1)
def printfoo():
  global foo 
  with foo.get_lock():
    print(foo.value)
if __name__ == '__main__':
  with foo.get_lock():
    foo.value = 2
  Process(target=printfoo).start()

On Linux and MacOS, this displays the expected 2. On Windows, it displays 1, even though the modification to the global Value is made before the call to Process. How can I make the change visible to the child process on Windows, too?

Mark Reed
  • 91,912
  • 16
  • 138
  • 175
  • 3
    [Python Multiprocess diff between Windows and Linux](https://stackoverflow.com/a/6596695/1755108) – brokenfoot Feb 28 '20 at 02:21
  • @brokenfoot: That's not really a great link. The questioner knows that multiprocessing behaves differently on Windows and Linux, and they've explicitly used the mechanisms available to share data between processes. They just used the mechanisms wrong. – user2357112 Feb 28 '20 at 02:43
  • If I used the mechanisms wrong, @user2357112supportsMonica, could you point me at how to use them right? I tried wrapping the Value access with calls to its `get_lock` method and that did not change the result. – Mark Reed Feb 28 '20 at 03:11
  • Your new edit causes an entirely different problem - you're trying to get the lock of an object that doesn't exist yet. That'll just throw a NameError before you even get to the worker process creation. – user2357112 Feb 28 '20 at 03:23

1 Answers1

2

The problem here is that your child process creates a new shared value, rather than using the one the parent created. Your parent process needs to explicitly send the Value to the child, for example, as an argument to the target function:

from multiprocessing import Process, Value

def use_shared_value(val):
    val.value = 2

if __name__ == '__main__':
    val = Value('i', 1)
    p = Process(target=use_shared_value, args=(val,))
    p.start()
    p.join()
    print(val.value)

(Unfortunately, I don't have a Windows Python install to test this on.)

Child processes cannot inherit globals on Windows, regardless of whether those globals are initialized to multiprocessing.Value instances. multiprocessing.Value does not change the fact that the child re-executes your file, and re-executing the Value construction doesn't use the shared resources the parent allocated.

user2357112
  • 260,549
  • 28
  • 431
  • 505
  • Yeah, I knew I could pass the value as an argument to the children, but I was hoping to avoid that since it's a breaking internal-API change. It doesn't look like I can avoid it, though. Even if I do use a shared Value, I will still need a separate Lock and would just wind up passing _that_ as a new argument, so it's the same change either way. Also, the child never modifies the value for the parent's benefit, so it doesn't have to be a Value at all; that's why it was originally just a regular global variable, which the child inherits on POSIX OSes – but not on Windows. – Mark Reed Feb 28 '20 at 03:26