1

I have a Python 2.7 program that converts audiobook files. Since there's a bunch (100-300) of flac files, and I have a shiny, new computer with an i7 processor, I'm working on a multiprocessing version of the program.

I get the list of files using glob, then create a list of tuples...

fis = []
for fi in globlist:
  fis.append[(fi, author, title, year, genre)]

I use pool.map(convert, fis) and the program works. WAY faster than the old one-at-a-time version.

Is there a way to create a tuple (author, title, year, genre) in the main function and have it available to the convert function? A dictionary would be fine, too. It doesn't work if I use global variables because I guess child-processes don't inherit anything from the parent in Windows. I'd just rather not have several hundred copies of it in the files list, or waste time recreating it in every child-process.

Thanks.

user3456239
  • 106
  • 1
  • 3
  • You could use `functools.partial()` to avoid passing the same arguments multiple times. See [Python multiprocessing pool.map for multiple arguments](http://stackoverflow.com/q/5442910/4279) – jfs Apr 01 '14 at 21:14

1 Answers1

2

You can try to use an initializer function for Pool(). For instance it could be something like that:

import multiprocessing as mp

def initializer(myvariable):
    global myvariable

if __name__ == "__main__":
    fis = []
    for fi in globlist:
        fis.append[(fi, author, title, year, genre)]
    with mp.Pool(initializer=initializer, initargs=fis) as pool:
        pool.map(convert, fis)

Now convert should be able to see a variable called myvariable which would be the fis list in the present case. The reason this should work (I have not tested the code as I do not have a Windows python installation at hand) is that each spawned process runs the initializer. Hope it helps.

Med
  • 21
  • 3
  • 1
    It is a `SyntaxError` when both parameter and global have the same name, use assignment instead: `myvariable = myvariable_`. `mp.Pool` is not a context manager in Python 2.7 – jfs Apr 01 '14 at 21:13