How do I make a Manager list dictionary that can be shared between multiple processes?

Question

I want to share a list of dictionaries between multiple processes like the following:

msgs = [{ 1: "A", 2: "B"}, 
        {20 : "AAA", 30 : "BBB"}, 
        {100 : "777", 200 : "888"}]

I've looked at the other posts regarding how to use Manager dicts but they don't show to to make a list of another type (dict in this case)

For example: how can I share a dictionary across multiple processes?

^Unfortunately, I wasn't able to extend that to what I want. Also, I am not using a pool of processes. I have 3 separate processes that I've instantiated.

Would anyone be able to show me an example of how to make a list of dictionaries that can be shared across processes?

score 0 · Accepted Answer · answered Oct 23 '19 at 18:55

If you want to share your dictionaries within the list you have to not only create the list but also the dictionaries via Manager(). For example

msg = manager.list()
msg.append(manager.dict({ 1: "A", 2: "B"}))

Otherwise the changes to your dictionary inside the process will not be registered. You might want to look a this answer for details.

An extended version of the linked example could look something like this

from multiprocessing import Process, Manager

def f(msg):
    for ele in msg:
        if 1 in ele:
            ele[1] += "hi"
        if 100 in ele:
            ele[100] += "5"

if __name__ == '__main__':
    manager = Manager()

    msg = manager.list()
    msg.append(manager.dict({ 1: "A", 2: "B"}))
    msg.append(manager.dict({20 : "AAA", 30 : "BBB"}))
    msg.append(manager.dict({100 : "777", 200 : "888"}))

    p1 = Process(target=f, args=(msg,))
    p2 = Process(target=f, args=(msg,))
    p3 = Process(target=f, args=(msg,))

    p1.start()
    p2.start()
    p3.start()
    p1.join()
    p2.join()
    p3.join()

    for d in msg:
        print(str(d))

Resulting in the output

{1: 'Ahihihi', 2: 'B'}

{20: 'AAA', 30: 'BBB'}

{100: '777555', 200:'888'}

So does this also mean that changes made any to the dictionaries in any of the processes would also be seen by the other processes? I basically want a "global list dictionary" that can be changed by any of the processes. — John, Oct 25 '19 at 16:33

How do I make a Manager list dictionary that can be shared between multiple processes?

1 Answers1