Python multiprocessing not working when time.sleep is used

Question

I have one function which will take different inputs and I want that function to run parallely. Below is what I tried , but it's not working due to time.sleep I think.

from multiprocessing import Process
from time import sleep
import time

def f(name):
    print('hello', name)
    time.sleep(10)

l1 = Queue()
a = Process(target=f('Tom'))
a.start()

l2 = Queue()
b = Process(target=f("Stock"))
b.start()

print (l1.get())
print (l2.get())

I want the function to run parallely. Currently the function waits for 10 seconds before it goes to the second execution.

You are *calling the function* then passing the result to `Process`, in this case, `None`, right here: `a = Process(target=f('Tom'))` [Look at the multiprocessing docs](https://docs.python.org/3/library/multiprocessing.html#multiprocessing.Process), what does it say about the value you are supposed to pass to `target`? — juanpa.arrivillaga, Sep 23 '21 at 06:47
Does this answer your question? [thread starts running before calling Thread.start](https://stackoverflow.com/questions/11792629/thread-starts-running-before-calling-thread-start) — MisterMiyagi, Sep 23 '21 at 08:12

Corralien · Answer 1 · 2021-09-23T07:43:45.967

0

Use Pool and starmap:

import multiprocessing
import time

def f(name, place):
    print('hello', name, place)
    time.sleep(5)

if __name__ == '__main__':
    data = [('Louis', 'val1'), ('Paul', 'val2'), ('Alexandre', 'val3'),
            ('John', 'val4'), ('Tom', 'val5'), ('Bob', 'val6')]
    with multiprocessing.Pool(2) as pool:
        pool.starmap(f, data)

Read carefully the multiprocessing guidlines

Safe importing of main module

Make sure that the main module can be safely imported by a new Python interpreter without causing unintended side effects (such a starting a new process).

edited Sep 23 '21 at 07:43

answered Sep 23 '21 at 06:51

Corralien

109,409
8
28
52

I think more than anything, `__main__` is the important part. – Peter Wood Sep 23 '21 at 06:59
@PeterWood Thanks for the reply . Assuming the function takes 2 input arguments , `def f(name , place )` . In this scenario how can we implement the multiprocessing. – Meenakshisundaram Ramanathan Sep 23 '21 at 07:22
This answer just seems to avoid the problem. The error made in the question code is an important one to understand. Merely showing how a *different* approach looks when done correctly doesn't seem to cover that. – MisterMiyagi Sep 23 '21 at 07:35
@MisterMiyagi. I don't avoid the problem. The question is *I want the function to run parallely.*. Does my solution not answer to that? – Corralien Sep 23 '21 at 07:45
@MeenakshisundaramRamanathan. I updated my question and replace `map` by `starmap`. I replace `names` by a list of tuples. – Corralien Sep 23 '21 at 07:47
@Corralien The question doesn't ask anything in specific. What it does do is show an error from a small but severe misunderstanding. I think showing a different approach is fine, but doing so without any explanation on *what* the issue was or *why* the other code fixes it isn't helpful. – MisterMiyagi Sep 23 '21 at 07:56

Booboo · Accepted Answer · 2021-09-23T11:24:01.780

See the comment posted by @juanpa.arrivillaga. Just specify as the target argument of the Process constructor the name of the function; do not call the function. The arguments to your target are specified separately as the args argument, either as a tuple or a list.

Also, since f is not returning a useful value, there is no reason to have Queue instances for the results. In any case, you are not passing the Queue instances to f and nothing is being put to the queue so I can't understand why you would be attempting to issue calls to get against these Queue instances; these calls will hang forever. The argument you do need to be passing to f is the name, as follows:

from multiprocessing import Process
#from time import sleep
import time

def f(name):
    print('hello', name)
    time.sleep(10)

start_time = time.time()

a = Process(target=f, args=('Tom',))
a.start()

b = Process(target=f, args=("Stock",))
b.start()

# Wait for processes to complete
a.join()
b.join()

elapsed_time = time.time() - start_time
print(elapsed_time)

Prints:

hello Tom
hello Stock
10.212252855300903

If you were running this on a platform such as Windows, then you would need to place the process-creating code in a special block as follows;

from multiprocessing import Process
#from time import sleep
import time

def f(name):
    print('hello', name)
    time.sleep(10)

# Required on platforms that create processes with the "spawn" method:
if __name__ == '__main__':
    start_time = time.time()

    a = Process(target=f, args=('Tom',))
    a.start()

    b = Process(target=f, args=("Stock",))
    b.start()

    # Wait for processes to complete
    a.join()
    b.join()

    elapsed_time = time.time() - start_time
    print(elapsed_time)

Python multiprocessing not working when time.sleep is used

2 Answers2