Terminating multiple processes containing while loops upon user input

Question

This was created with a combination of:

The idea of the code is that two functions with while loops doing something are run at the same time and it can be stopped when the user presses enter.

import sys, select, os, datetime
from multiprocessing import Process
from time import time

def func_control():
    global switch
    switch = 1
    while True:
        #os.system('cls' if os.name == 'nt' else 'clear')
        print "I'm doing stuff. Press Enter to stop me!"
        if sys.stdin in select.select([sys.stdin], [], [], 0)[0]:
            line = raw_input()
            switch = 0
            break

def func_1():
    global i1
    i1 = 0
    while switch != 0:
        i1 = i1+1
    return 0

def func_2():
    global i2
    i2 = 0
    while switch != 0:
        i2 = i2+1
    return 0

def start_fcontrol():
    while time() < start_time: pass
    func_control()

def start_f1():
    while time() < start_time: pass
    func_1()

def start_f2():
    while time() < start_time: pass
    func_2()        

switch = 1
i1 = 0
i2 = 0
procs = []
procs.append(Process(target=start_fcontrol))
procs.append(Process(target=start_f1))
procs.append(Process(target=start_f2))
start_time = time() + 1
map(lambda x: x.start(), procs)
map(lambda x: x.join(), procs)
print "i1 = ", i1
print "i2 = ", i2

The code throws out the following error which does not happen when using the code supplied in the first link by itself:

I'm doing stuff. Press Enter to stop me!
Process Process-1:
Traceback (most recent call last):
  File "/usr/lib/python2.6/multiprocessing/process.py", line 232, in _bootstrap
    self.run()
  File "/usr/lib/python2.6/multiprocessing/process.py", line 88, in run
    self._target(*self._args, **self._kwargs)
  File "atest.py", line 31, in start_fcontrol
    func_control()
  File "atest.py", line 12, in func_control
    line = raw_input()
EOFError: EOF when reading a line

I've tried to keep the code as clean and as minimal as possible.

Well, one thing to note for sure is that `global`s don't span across processes. They will work across threads since threads share the same interpreter, but you definitely can't achieve anything close to truly synchronous execution with threads because of the GIL. If you want to pass data between processes, you need to use the `multiprocessing` data passing mechanisms, like queues, locks, and managers. A crash course in synchronous multithreaded programming is going to take a bit more than an hour though, I'm afraid... — Silas Ray, Mar 13 '14 at 22:22
@SilasRay Thank you for your reply. Can you suggest a way to restructure this then? Perhaps with the capture of user input elsewhere? — user3394391, Mar 13 '14 at 22:29
Globals might not span between processes, but the filesystem does, assuming it's all one computer. Have all the processes periodically check for "shut_down_everything.txt" and stop if they find it. Have the user input create it. — TessellatingHeckler, Mar 13 '14 at 22:34
I won't lie it sounds a bit rough and ready but if its the only thing that will work I'll give it a go! Any idea why Im getting the EOF error here though? Will a new process still use the same imported modules in the main program? — user3394391, Mar 13 '14 at 22:37

score 2 · Accepted Answer · answered Mar 13 '14 at 22:59

Ugly, but it works. Synchronization is done via shared memory. And you need to pass stdin file descriptor to the child process and open it there.

import sys, select, os, datetime
from multiprocessing import Process, Value
from time import time

def func_control(fileno, switch):
    sys.stdin = os.fdopen(fileno)  #open stdin in this process

    print "I'm doing stuff. Press Enter to stop me!"
    some_str = raw_input("> ")
    switch.value = 0
    return

def func_1(i1, switch):
    i1.value = 0
    while switch.value != 0:
        i1.value = i1.value+1
    return 0

def func_2(i2, switch):
    i2.value = 0
    while switch.value != 0:
        i2.value = i2.value+1
    return 0

def start_fcontrol(start_time, fn, switch):
    while time() < start_time.value: pass
    func_control(fn, switch)

def start_f1(i1, start_time, switch):
    while time() < start_time.value: pass
    func_1(i1, switch)

def start_f2(i2, start_time, switch):
    while time() < start_time.value: pass
    func_2(i2, switch)        

switch = Value('i', 1)
i1 = Value('i', 0)
i2 = Value('i', 0)
start_time = Value('d', time() + 1)

fn = sys.stdin.fileno() #get original file descriptor

procs = []
procs.append(Process(target=start_fcontrol, args=(start_time, fn, switch)))
procs.append(Process(target=start_f1, args=(i1, start_time, switch)))
procs.append(Process(target=start_f2, args=(i2, start_time, switch)))
map(lambda x: x.start(), procs)
map(lambda x: x.join(), procs)
print "i1 = ", i1.value
print "i2 = ", i2.value

LOVE IT! Not as elegant as the other solution posted but it outperforms it in terms of synchronicity. — user3394391, Mar 13 '14 at 23:53

score 1 · Answer 2 · answered Mar 13 '14 at 22:37

As @Silas Ray said, you cannot count on these globals when you have different processes. Maybe you should use Threads. It is as simple as what you've done but with more bennefits for you. See the example. I've tested and it seems to work!

Very important is that you mark the Threads as daemon, which means that they are going to die when the main process stops! :-)

from threading import Thread

def func_control():
    global switch
    switch = 1
    while True:
        #os.system('cls' if os.name == 'nt' else 'clear')
        print "I'm doing stuff. Press Enter to stop me!"
        if sys.stdin in select.select([sys.stdin], [], [], 0)[0]:
            line = raw_input()
            switch = 0
            break

def func_1():
    global i1
    i1 = 0
    while switch != 0:
        i1 = i1+1
    return 0

def func_2():
    global i2
    i2 = 0
    while switch != 0:
        i2 = i2+1
    return 0

switch = 1
i1 = 0
i2 = 0

myThread1 = Thread(target=func_1)
myThread1.daemon = True

myThread2 = Thread(target=func_2)
myThread2.daemon = True

myThread1.start()
myThread2.start()

func_control()

Yes. You will notice immediately if you look at the CPU load. I have tested it. — Javier, Mar 13 '14 at 22:51
Is there a method to minimise the time delay between threads 1 and 2 like in the original method and also the second link on my original post? Can you do this in one line? "myThread1 = Thread(target=func_1) myThread1.daemon = True" — user3394391, Mar 13 '14 at 23:08
In addition I just read that daemon threads can screw things up with regards to open files, etc? This is important as my while loops will be manipulating files. — user3394391, Mar 13 '14 at 23:12

Terminating multiple processes containing while loops upon user input

2 Answers2