2

I have this test code using "concurrent.futures.ProcessPoolExecutor() as executor" and tried many combinations, but I can't get it it to work. I am certainly missing some syntax.

What this should be doing is share the x variable across the two functions and foo2 prints out the modified one.

def foo(x):
    while True:
        time.sleep(1)
        print('foo')
        x.value += 1

def foo2(y):
    while True:
        time.sleep(1.5)
        print('foo2')
        print(y.value)

def main():
    x = Value('i')

    with concurrent.futures.ProcessPoolExecutor() as executor:    
        executor.submit(foo, x) 
        executor.submit(foo2, x)

if __name__ == '__main__':
    main()

I tried these combos, but all not working:

(foo(x), x) 
(foo2(x), x) 


(foo(x)) 
(foo2(x)) 

(foo(x), x) 
(foo2(x), x) 

(foo, x) 
(foo2, x) 

(foo, x) 
(foo2, x)

when I try these both combinations I get at least some (unshared variable) output:

x += 1
print(x)

(foo, x,value) 
(foo2, x.value)

Output:

foo
foo2
0
foo
foo
foo2
0

I then modified a different script and got this to work:

def foo(y, name=''):
    while True:
        time.sleep(1)
        y.value += 1
        # print(y.value, name)

def foo2(y, name=''):
    while True:
        time.sleep(0.5)
        print(y.value)
    
if __name__ == "__main__":

    y = Value('i')
    
    Process( target=foo, args=(y, 'foo') ).start()
    Process( target=foo2, args=(y, 'foo2') ).start()

Output:

0
1
1
2
2
3

Really wanting it to make it work using the "executor.submit" feature. I also don't understand why the "Process" feature strictly requires 'foo' and name='' (otherwise it work work either)?

I am sure there is more to learn here! Any pointers greatly appreciated! I suspect some syntax error.

Dalalama231123
  • 101
  • 1
  • 1
  • 7
  • The submit function returns a `Future`. What happens when you call the `result` method on it? https://docs.python.org/3/library/concurrent.futures.html#concurrent.futures.Executor.submit – John Mee Aug 11 '22 at 22:50
  • 1
    Does it work if you use a `ThreadPoolExecutor` instead? My understanding is that when you run a function in a `ProcessPoolExecutor`, the arguments are pickled and sent to the other process -- that means that if the other process mutates them, you're not going to see the mutation in the object that you originally passed in. To get the result you'd need to `return` it from the other function so the caller can get it from the `Future` that's completed and returned by the other process. – Samwise Aug 11 '22 at 22:51
  • Thanks. I did "print(f1.result())" and got "RuntimeError: Synchronized objects should only be shared between processes through inheritance" – Dalalama231123 Aug 11 '22 at 22:55
  • You can PASS data to a process, but it becomes a completely separate object in a completely separate memory space. If you truly want to share a variable, you need to use a `multiprocessing.Manager`, which sets up a socket connection to communicate. You might consider if your communications can be handled by a Queue instead. – Tim Roberts Aug 11 '22 at 23:02
  • Thanks! Is the "Concurrent Execution" maybe not suited for indefinite loops? – Dalalama231123 Aug 11 '22 at 23:10
  • See [this answer](https://stackoverflow.com/a/35394592/75033). I don't think the looping is the problem. – John Mee Aug 11 '22 at 23:13
  • Does this answer your question? [ProcessPoolExecutor and Lock in Python](https://stackoverflow.com/questions/35394373/processpoolexecutor-and-lock-in-python) – John Mee Aug 12 '22 at 04:20

0 Answers0