3

I have some code which runs in parallel using the multiprocessing Pool class. Unfortunately some of the functions I use from another library have some verbose output. To abstract the problem have a look at the following example:

from multiprocessing import Pool

def f(x):
    print 'hello'
    return x*x

p = Pool(10)
p.map(f, [1,2,3])

So this will print 'hello' like 10 times. Is there a possibility to mute the output of the processes or to put the std.out in some variable? Thanks for any suggestion.

EDIT: I don't want to redirect the whole stdout, just for the processes of my pool.

Cœur
  • 37,241
  • 25
  • 195
  • 267
jrsm
  • 1,595
  • 2
  • 18
  • 39
  • possible duplicate of [Redirect stdout to a file in Python?](http://stackoverflow.com/questions/4675728/redirect-stdout-to-a-file-in-python) – Ami Tavory Jun 14 '15 at 13:29
  • what about [this one](http://stackoverflow.com/a/2829036/2395605)? It suggests a contextmanager. – Dave J Jun 14 '15 at 13:32
  • It doesn't work I think the problem is that multiple independent processes are spawned. – jrsm Jun 14 '15 at 13:36

1 Answers1

9

Use the initializer parameter to call mute in the worker processes:

import sys
import os
from multiprocessing import Pool

def mute():
    sys.stdout = open(os.devnull, 'w')    

def f(x):
    print 'hello'
    return x*x

if __name__ == '__main__':
    p = Pool(10, initializer=mute)
    p.map(f, [1,2,3])
    print('Hello')

prints

Hello
unutbu
  • 842,883
  • 184
  • 1,785
  • 1,677