Updated variable into multiprocessing python

Question

I use 2 python process and i wonder how share and update a variable. I manage to send variable to a process but this variable isn't updated during the process.

In my code, when the process worker is launched, it increase the variable a every 3 sec. In the same time the process my_service show continuously the value of a.

#!/usr/bin/python
# -*- coding: utf-8 -*-

#import multiprocessing as mp
#from multiprocessing import Process
import multiprocessing

import time
from globalvar import *
a=8
#toto=8

def worker():
    name = multiprocessing.current_process().name
    # print (name,"Starting")
    # time.sleep(2)
    # print (name, "Exiting")
    for a in range(1,4):
        print ("worker=",a)
        time.sleep(3)

def my_service(az):
    name = multiprocessing.current_process().name
    # print (name,"Starting")
    # time.sleep(3)
    # print (name, "Exiting")
    while True:
        print ("my_service=",az)
        time.sleep(2)

if __name__ == '__main__':
    #Process(target=worker).start()
    service = multiprocessing.Process(name='my_service', target=my_service,args=(a,))
    worker_1 = multiprocessing.Process(name='worker 1', target=worker)
    worker_2 = multiprocessing.Process(target=worker) # use default name

    worker_1.start()
    worker_2.start()
    service.start()

But the result isn't what i expect:

worker= 1
worker= 1
my_service= 8
my_service= 8
worker= 2
worker= 2
my_service= 8
worker= 3
worker= 3
my_service= 8

The variable into worker is increased, but the variable isn't shown uptaded in process service

So how to share updated varaible between process?

Thx,

The a in your workers and the a in your service are in entirely different scopes and are therefore not related at all and would not be expected to update each other. Even if you weren't doing this multiprocess, you wouldn't see them modify each other. — Mike A., Dec 17 '15 at 15:05
ok, but there's a solution to update variable between process ? — morbak, Dec 17 '15 at 15:09
Because it's multi-process and not multi-threaded, they variables aren't shared between processes, I believe. You could have each process do its work and then return the updated values to your main global process. — Mike A., Dec 17 '15 at 15:12
You might want to look into using the shared memory functionality of multiprocessing, like in: http://stackoverflow.com/questions/14124588/python-multiprocessing-shared-memory — Mike A., Dec 17 '15 at 15:16
So, do you think it's possible to share updated data with multi-threading? — morbak, Dec 17 '15 at 15:22
Yes. But this is a pretty complicated subject. Maybe you could start here: http://stackoverflow.com/questions/3044580/multiprocessing-vs-threading-python — Mike A., Dec 17 '15 at 15:23

score 6 · Answer 1 · edited May 23 '17 at 12:00

The issue in multiprocessing with python is that ever process is completely independant from the others. On lauching, it copies the current variables and then work on this copy : this means any modification of the state of a variable is not replicated to the other process. This is caused by the Python's Global Interpreter Lock, wich ensure that only one process have access to a variable at the same time, to avoid corrupting memory. You can see more here : What is a global interpreter lock (GIL)?

Now for your particular issue, you can use shared variables.

from multiprocessing import Value
a=Value('f', 0.0) # create a shared float, initialised at 0
a.value # read the value
a.value=50 # modify the value

You need to declare a and pass it as argument of each process.

But as you "bypass" the GIL, you need to manage your own access to this variable, to avoid having 2 process trying to read/modify it at the same time. This is why every shared variable comes with a Lock, that allow accessing the variable.

a.acquire() #acquire the Lock, forbidding access to other processes.
a.value # read the value
a.value=50 # modify the value
a.release() # don't forget to release the lock, or else you will block everything.

Be aware that in case of error/exception, if the lock isn't released, the access to your variable will be lost forever. If this is an issue, add this:

try:
    a.acquire() #acquire the Lock, forbidding access to other processes.
    a.value # read the value
    a.value=50 # modify the value
    a.release() # don't forget to release the lock, or else you will block everything.
except Exception as e:
    print e
    a.release()

Your final code :

#!/usr/bin/python
# -*- coding: utf-8 -*-

import multiprocessing
from multiprocessing import Value

import time
#from globalvar import *
a=Value('f', 8)
#toto=8

def worker(a):
    try:
        name = multiprocessing.current_process().name
        for i in range(1,4):
            a.acquire()
            a.value=i
            a.release()
            print ("worker=",a.value)
            time.sleep(3)
    except Exception as e:
        print e
        a.release()

def my_service(az):
    name = multiprocessing.current_process().name
    # print (name,"Starting")
    # time.sleep(3)
    # print (name, "Exiting")
    while True:
        try:
            az.acquire()
            print ("my_service=",az.value)
            az.release()
            time.sleep(2)
        except Exception as e:
            print e
            az.release()

if __name__ == '__main__':
    #Process(target=worker).start()
    service = multiprocessing.Process(name='my_service', target=my_service,args=(a,))
    worker_1 = multiprocessing.Process(name='worker 1', target=worker,args=(a,))
    worker_2 = multiprocessing.Process(target=worker,args=(a,)) # use default name

    worker_1.start()
    worker_2.start()
    service.start()

Updated variable into multiprocessing python

1 Answers1