multiprocessing inside exec command

Question

I need to use multiprocessing in a piece of code which is injected as string into exec function:

code = """
from multiprocessing import Process

def f(name):
    print('hello', name)

p = Process(target=f, args=('bob',))
p.start()
p.join()
"""

if __name__ == '__main__':
   exec(code)

I get the following error:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Programming\WinPython-64bit-3.3.5.0\python-3.3.5.amd64\lib\multiprocessing\forking.py", line 351, in main
self = load(from_parent)
AttributeError: 'module' object has no attribute 'f'

I am very new to multiprocessing... and I must say I have no idea what is wrong. My system is Windows 7 64-bit.

UPDATE: ... and a more general question: is it possible to run a user defined script (stored in a string) asynchronously in another process? This is actually what I try to achieve.

This must be windows specific. Your code works on mac/linux python 2.7 and 3.4. — jwillis0720, Jan 04 '15 at 00:15

score 1 · Answer 1 · edited May 23 '17 at 10:27

1

I think you might be running into the same indentation problem as seen here: Python AttributeError: Object has no attribute Try ensuring that your print and exec lines are properly tabbed.

edited May 23 '17 at 10:27

Community

1
1

answered Jan 04 '15 at 00:24

Joe Young

5,749
3
28
27

score 0 · Accepted Answer · answered Jan 04 '15 at 00:23

0

multiprocessing requires that the target function be importable by subprocesses from your __main__ module. Here, said module defines no f, therefore the constraint is not met (even though some operating systems manage to let you relax the constraint, it's terrible practice, and kills the portability of your code).

I would recommend analyzing the code string, e.g via the compile built-in, and ensuring a __main__-level definition of the required target.

answered Jan 04 '15 at 00:23

Alex Martelli

854,459
170
1,222
1,395

I am not sure if I fully understand what it means. Is there a way I can make it work just by changing the contents of the string in variable `code`? Suppose I cannot change anything outside of it. – HiFile.app - best file manager Jan 04 '15 at 00:49
No, unless you can set `f` as a global attribute of `__main__`, which exclusively changing the string `code` per se cannot do, then you can't make the whole code reliable for `multiprocessing` across all Python implementations. – Alex Martelli Jan 04 '15 at 01:10
OK, bad luck for me. :( I have to find a workaround for my problem. I have started playing with Pool and apply_async and put the multiprocessing ability outsize of the code string... seems promising so far. – HiFile.app - best file manager Jan 04 '15 at 10:24

score 0 · Answer 3 · answered May 19 '21 at 19:30

This is an old issue... In any case, the following works:

script = """
from multiprocessing import Process

def f(name):
    print('hello', name)

if __name__ == '__main__':
    p = Process(target=f, args=('bob',))
    p.start()
    p.join()
"""
exec(script)

multiprocessing inside exec command

3 Answers3