An attempt has been made to start a new process before the current process has finished its bootstrapping phase in python

Question

I try to run this code on python

import multiprocessing

manager = multiprocessing.Manager()
final_list = manager.list()

input_list_one = ['one', 'two', 'three', 'four', 'five']
input_list_two = ['six', 'seven', 'eight', 'nine', 'ten']

def worker(data):
    for item in data:
        final_list.append(item)

if __name__ == '__main__':    

    process1 = multiprocessing.Process(target=worker, args=[input_list_one])
    process2 = multiprocessing.Process(target=worker, args=[input_list_two])

    process1.start()
    process2.start()
    process1.join()
    process2.join()

    print(final_list)

But this error is happened: RuntimeError:

An attempt has been made to start a new process before the
current process has finished its bootstrapping phase.

This probably means that you are not using fork to start your
child processes and you have forgotten to use the proper idiom
in the main module:

    if __name__ == '__main__':
        freeze_support()
        ...

The "freeze_support()" line can be omitted if the program
is not going to be frozen to produce an executable.

Booboo · Accepted Answer · 2021-10-10T20:59:14.737

The problems is with the statement:

manager = multiprocessing.Manager()

which does its "magic" by starting a server process. Therefore, this statement needs to be moved to within the if __name__ = '__main__': block along with the creation of the managed list, which now needs to be passed as an additional argument to your process function, worker. In fact, you might as well move all declarations at global scope that do not really need to be there within the if __name__ = '__main__': block for efficiency because they would otherwise be needlessly executed by each new process created.

import multiprocessing

def worker(final_list, data):
    for item in data:
        final_list.append(item)

if __name__ == '__main__':
    manager = multiprocessing.Manager()
    final_list = manager.list()

    input_list_one = ['one', 'two', 'three', 'four', 'five']
    input_list_two = ['six', 'seven', 'eight', 'nine', 'ten']

    process1 = multiprocessing.Process(target=worker, args=[final_list, input_list_one])
    process2 = multiprocessing.Process(target=worker, args=[final_list, input_list_two])

    process1.start()
    process2.start()
    process1.join()
    process2.join()

    print(final_list)

Prints:

['six', 'seven', 'eight', 'nine', 'one', 'ten', 'two', 'three', 'four', 'five']

Let me elaborate a bit on my answer:

You are clearly running on a platform that uses the spawn method to create new processes. This means that to launch a new process a new, empty address space is created and a new instance of the Python interpreter is run against the source. Before the target of your Process instance is invoked, any statement in the source file that is at global scope will first be executed to initialize the process, except any statement within a if __name__ = '__main__': block (because the __name__ value will not be '__main__' for the new process). This is why you need to put code that creates new processes within such a block, i.e. to avoid getting into what would be a recursive loop creating new processes ad infinitum if this went undetected. In your case it did not go undetected and you got the error message you saw.

But even if creating a Manager instance had not resulted in a new process getting created, your program would not have been correct. Having the statement final_list = manager.list() at global scope meant that all three processes in your running program would have been accessing three different instances of final_list.

As you said in your answer this code creates 3 separates instance of final_list .My question is how can I code this part to just create one instance in memory for better performance? — Ehsan Akbar, Oct 11 '21 at 05:24
Recently I have tried to implement LMAX system in python :https://stackoverflow.com/questions/69434004/business-logic-imp-in-lmax-architecture-in-python — Ehsan Akbar, Oct 11 '21 at 05:26
Your original code, if you did not have the recursive process-creation issue, would have created 3 instances of a managed list (i.e. `final_list`). My code above only creates one in the main process, which is then passed to the other two new processes (actually, `final_list` is a *proxy* to the actual list this is implemented by the server process created by the `SyncManager` instance returned by the call to `Manager()`). — Booboo, Oct 11 '21 at 10:25
could you please take a look this question :https://stackoverflow.com/questions/69521484/attributeerror-cant-get-attribute-journalerreader-on-module-mp-main — Ehsan Akbar, Oct 11 '21 at 11:04

score 0 · Answer 2 · answered Jun 30 '22 at 08:41

0

It happens in windows spawn.py file. As it is Runtime error, we can ignore it when starting the process, using "if name == 'main':" EG: yolo v5 with python 3.8.5

if __name__ == '__main__':
    from yolov5 import train
    train.run()

answered Jun 30 '22 at 08:41

Biman Pal

391
4
9

An attempt has been made to start a new process before the current process has finished its bootstrapping phase in python

2 Answers2