Why to call multiprocessing module with Process can create same instance?

Question

My platform info:

uname -a
Linux debian 5.10.0-9-amd64 #1 SMP Debian 5.10.70-1 (2021-09-30) x86_64 GNU/Linux
python3 --version
Python 3.9.2

Note:to add a lock can make Singleton class take effect ,that is not my issue,no need to talk the process lock.
The same parts which will be executed in different status:

Singleton class

class Singleton(object):
    def __init__(self):
        pass
    @classmethod
    def instance(cls, *args, **kwargs):
        if not hasattr(Singleton, "_instance"):
            Singleton._instance = Singleton(*args, **kwargs)
        return Singleton._instance

working function in the process

import time,os
def task():
    print("start the process %d" % os.getpid())
    time.sleep(2)
    obj = Singleton.instance()
    print(hex(id(obj)))
    print("end the process  %d" % os.getpid())

Creating multi-process with processing pool way:

from multiprocessing.pool import Pool    
with Pool(processes = 4) as pool:
    [pool.apply_async(func=task) for item in range(4)]
    #same effcet with pool.apply ,pool.map,pool.map_async in this example,i have verified,you can try again
    pool.close()
    pool.join()

The result is as below:

start the process 11986
start the process 11987
start the process 11988
start the process 11989
0x7fd8764e04c0
end the process  11986
0x7fd8764e05b0
end the process  11987
0x7fd8764e0790
end the process  11989
0x7fd8764e06a0
end the process  11988

All sub-process has its own memory,they don't share space each other,they don't know other process has already created an instance,so it output different instances.

Creating multi-process with Process way:

import multiprocessing
for i in range(4):
    t = multiprocessing.Process(target=task)
    t.start()

The result is as below:

start the process 12012
start the process 12013
start the process 12014
start the process 12015
0x7fb288c21910
0x7fb288c21910
end the process  12014
end the process  12012
0x7fb288c21910
end the process  12013
0x7fb288c21910
end the process  12015

Why it create same instance with this way ?What is the working principle in the multiprocessing module?
@Reed Jones,i have read the related post you provided for many times. In lmjohns3's answer:

So the net result is actually the same, but in the first case you're guaranteed to run foo and bar on different processes.

The first case is Process sub-module,the Process will guarante to run on different processes,so in my case :

import multiprocessing
for i in range(4):
    t = multiprocessing.Process(target=task)
    t.start()

It should result in several (maybe 4 or not,at least bigger than 1) instances instead of the same instance.

I am sure that the material can't explain my case.

It doesn't look to me like the objects are _actually_ the same: Setting an attribute on one of them isn't seen by the other processes. Maybe the way a Python process gets its memory addresses is just so consistent, that it will get the same addresses (which of course map to _different_ addresses in actual RAM)? (maybe one of the interfaces uses some randomness which influences this?) — L3viathan, Oct 17 '21 at 07:43

noxdafox · Accepted Answer · 2021-10-18T11:07:20.083

As already explained in this answer, id implementation is platform specific and is not a good method to guarantee unique identifiers across multiple processes.

In CPython specifically, id returns the pointer of the object within its own process address space. Most of modern OSes abstract the computer memory using a methodology known as Virtual Memory.

What you are observing are actual different objects. Nevertheless, they appear to have the same identifiers as each process allocated that object in the same offset of its own memory address space.

The reason why this does not happen in the pool is most likely due to the fact the Pool is allocating several resources in the worker process (pipes, counters, etc..) before running the task function. Hence, it randomizes the process address space utilization enough such that the object IDs appear different across their sibling processes.

Why to call multiprocessing module with Process can create same instance?

1 Answers1