3

I have a following simple code

from multiprocessing import Pool
x = []


def func(a):
    print(x,a)


def main():
    a = [1,2,3,4,5]
    pool = Pool(1)
    global x
    x = [1,2,3,4]
    ans = pool.map(func,a)
    print(x)

It gives me the result

[] 1
[] 2
[] 3
[] 4
[] 5
[1, 2, 3, 4]

I expected the result to reflects the change in global variable x. Which seems that the changed in global variable x is not updated before the pool call. I would like to ask what is the cause of this?

hoan duc
  • 70
  • 10
  • why do you think x is being used in pool? you don't use `x` with pool in any way here. – Mike - SMT Aug 15 '17 at 14:50
  • In function "func" I just print the value of "x". – hoan duc Aug 15 '17 at 14:53
  • `Which seems that the global variable x is not used in the pool at all`: This is because x is not used in pool at all. This is what I am getting at. Maybe your wording is misleading to your problem. – Mike - SMT Aug 15 '17 at 14:55
  • Thank you, I have edited the question. – hoan duc Aug 15 '17 at 14:59
  • Possible duplicate of [Python multiprocessing global variable updates not returned to parent](https://stackoverflow.com/questions/11055303/python-multiprocessing-global-variable-updates-not-returned-to-parent) – noxdafox Aug 15 '17 at 15:16

3 Answers3

1

So I have done what GuangshengZuo suggested, and sadly the result was not desirable. After looking deeper into it, I realized the problem was not because of script, but rather the OS.

In windows, there is no os.fork(), hence the change in global variable is not copied. But, on Unix machine, the script works fine.

hoan duc
  • 70
  • 10
0

Two seperate processes will not share the same global variables. A multiprocessing pool abstracts away the fact that you are using two seperate processes which makes this tough to recognise.

pypypy
  • 1,075
  • 8
  • 18
0

I think it is because this is multiprocess, not multithread. the main process and the new process does not share a same global variable. So the new process has the copy of the main process when x is [], and after created, main process change x's value, but it does not change to new process's x.

if change the code to this :

from multiprocessing import Pool
x = []


def func(a):
    print(x,a)


def main():
    a = [1,2,3,4,5]
    global x
    x = [1,2,3,4]
    pool = Pool(1)
    ans = pool.map(func,a)
    print(x)

and the ouput will be what you want. Notice the pool = Pool(1) 's position

GuangshengZuo
  • 4,447
  • 21
  • 27
  • Sadly I have the same result. Would you mind to check if things work with your computer? – hoan duc Aug 15 '17 at 15:35
  • I checked. and if the pool = Pool(1) is before x's change, the output is same as yours. and if I move this code down, the output will be ([1,2,3,4],i) .... And I run this on ubuntu 64 and python 2.7. – GuangshengZuo Aug 15 '17 at 15:40
  • This might be the cause of different version, I use python 2.7. – hoan duc Aug 15 '17 at 16:26
  • read carefully please, I use python 2.7 also. and I mentioned it. If you understand what happens, you will be supposed to different result. Hopefully you can find why your output is same and fix it. – GuangshengZuo Aug 16 '17 at 04:24