Simultaneous file writing using python multiprocessing

Question

I'm developing a project in which I use around 6 sensors feeding into a Beaglebone Black, which saves that data to 6 different files continuously. Through another SO question (https://stackoverflow.com/a/36634587/2615940), I learned that the multiprocessing module would do this for me, but when running my new code, I only obtain 1 file as opposed to 6. How can I modify this code to obtain the desired 6 results files?

*I've edited my file to include Manager according to skrrgwasme's suggestion below, but now the code runs, and produces nothing. No errors, no files. Just runs.

The code:

import Queue
import multiprocessing
import time

def emgacq(kill_queue, f_name, adcpin):
     with open(f_name, '+') as f:
        while True:
            try:
                val = kill_queue.get(block = False)
                if val == STOP:
                    return
            except Queue.Empty:
                pass
            an_val = ADC.read(adcpin) * 1.8
            f.write("{}\t{}\n".format(ms, an_val))
def main():    
    #Timing stuff
    start = time.time()
    elapsed_seconds = time.time() - start
    ms = elapsed_seconds * 1000

    #Multiprcessing settings
    pool = multiprocessing.Pool()
    m = multiprocessing.Manager()
    kill_queue = m.Queue()            

    #All the arguments we need run thru emgacq()
    arg_list = [
        (kill_queue, 'HamLeft', 'AIN1'),
        (kill_queue, 'HamRight', 'AIN2'),
        (kill_queue, 'QuadLeft', 'AIN3'),
        (kill_queue, 'QuadRight', 'AIN4'),
        (kill_queue, 'GastLeft', 'AIN5'),
        (kill_queue, 'GastRight',  'AIN6'),
        ]

    for a in arg_list:
        pool.apply_async(emgacq, args=a) 

    try: 
        while True:
            time.sleep(60) 
    except KeyboardInterrupt:
        for a in arg_list:
            kill_queue.put(STOP)
        pool.close()
        pool.join()
        raise f.close()

if __name__ == "__main__":
    main()

Answer 1

Your main issue is that your list of arguments for your subprocess functions is incorrect:

f_list = [
    emgacq(kill_queue, 'HamLeft', 'AIN1'), 
    # this calls the emgacq function right here - blocking the rest of your 
    # script's execution

Also, your apply_async call is wrong:

for f in f_list:
    pool.apply_async(f, args=(kill_queue)) 
    # f is not a function here - the arguments to the apply_async function
    # should be the one function you want to call followed by a tuple of 
    # arguments that should be provided to it

You want this, which also includes a manager for the queue (see https://stackoverflow.com/a/9928191/2615940) and puts all of your code in a main function:

# put your imports here
# followed by the definition of the emgacq function

def main():

    #Timing stuff
    start = time.time()
    elapsed_seconds = time.time() - start
    ms = elapsed_seconds * 1000

    pool = multiprocessing.Pool()
    m = multiprocessing.Manager()
    kill_queue = m.Queue()

    arg_list = [
            (kill_queue, 'HamLeft', 'AIN1'), 
            (kill_queue, 'HamRight', 'AIN2'),
            (kill_queue, 'QuadLeft', 'AIN3'),
            (kill_queue, 'QuadRight', 'AIN4'),
            (kill_queue, 'GastLeft', 'AIN5'),
            (kill_queue, 'GastRight',  'AIN6'),
        ]

    for a in arg_list:
        pool.apply_async(emgacq, args=a)
        # this will call the emgacq function with the arguments provided in "a"

if __name__ == "__main__":
    # you want to have all of your code in a function, because the workers
    # will start by importing the main module they are executing from,
    # and you don't want them to execute that code all over again
    main()