1

Intuitively, the output should always be 15. But sometimes, it generate 11 or 12 or something else. I tried add some delay, but didn't solve the problem.

from multiprocessing import Pool, Manager
import time

def func(dic, c):
    dic['count'] += c
    # time.sleep(0.1)

if __name__=="__main__":
    d = Manager().dict()    # a manager to enable data sharing b
    d['count'] = 0
    args = [(d,1), (d,2), (d,3), (d,4), (d,5)]
    pool = Pool(5)
    pool.starmap(func, args)   
    pool.close()
    pool.join()
    print(f'dic={d}')
Xian
  • 11
  • 1

2 Answers2

0

The increment operator += is not atomic. This means that it is not safe to use in this way. When you invoke this operator the value is read and then replaced by the incremented value. If a second process changes the value between these two operations that change will be will be lost.

See this answer: Is the += operator thread-safe in Python?

Simon Notley
  • 2,070
  • 3
  • 12
  • 18
0

Your problem is this statement:

dic['count'] += c

It reads your dictionary, then increments by c and stores the value but this is not an atomic operation. Another process may have changed the dictionary between read and write operations and then your write will "overwrite" the change that happened between the operations.

You can solve this by passing a lock and using it:

def func(dic, lock, c):
    with lock:
        dic['count'] += c
    # time.sleep(0.1)

if __name__=="__main__":

    d = Manager().dict()    # a manager to enable data sharing b
    l = Manager().Lock()
    d['count'] = 0
    args = [(d, l,1), (d,l,2), (d,l,3), (d,l,4), (d,l,5)]
    pool = Pool(5)
    pool.starmap(func, args)
    pool.close()
    pool.join()
    print(f'dic={d}')

This ensures the whole dictionary operation stays atomic.

Hannu
  • 11,685
  • 4
  • 35
  • 51