Not able to write into a file using Python multiprocessing

Question

from itertools import product

f = open('filename.txt', 'a')

def worker(i, j):
    print i,j
    f.write("%s\t%s\n"%(i,j))
    return

def main():
    a_list = ['1', '2', '3', '4', '5'] #5 item
    b_list = ['6', '7', '8'] #3 item
    # Total 5*3=15 combinations

    from multiprocessing import Pool
    pool = Pool(processes=4)
    results = [pool.apply_async(worker, args=(i, j)) for i, j in product(a_list, b_list)]
    output = [p.get() for p in results]

main()
f.close()

this is the code I'm trying to run and store result in a txt file but I'm unable to findout why this isn't writing, although its printing in terminal. any help would be appreciated.

Is this linux or windows? Does the content appear in the file after the script terminates? — tdelaney, Feb 04 '15 at 16:53
its linux(ubuntu). I get blank file after completion of this program. — micheal, Feb 04 '15 at 17:43

score 4 · Accepted Answer · answered Feb 04 '15 at 16:51

4

add f.flush() after the f.write(...) statement

answered Feb 04 '15 at 16:51

Vor

33,215
43
135
193

2

Thanks! Could you please explain me the issue because I've never face such situation. – micheal Feb 04 '15 at 17:25
1

@michael I believe the reason is that, because the file object isn't opened in your worker process, it doesn't automatically get flushed/closed when the worker exits. Because of this, your write never actually gets written to the file, it just gets written to a buffer local to the worker, which is then lost when the process exits. – dano Feb 04 '15 at 17:39

Not able to write into a file using Python multiprocessing

1 Answers1

Linked