If starting unix pipe with Parallel command, pipe commands complain after cntrl-c

Asked Sep 06 '15 at 17:31

Active Sep 06 '15 at 18:06

Viewed 1,039 times

If I start a process in parallel with joblib that calls a subprocess with a unix pipe (sed | uniq | blabla) many of those processes write error messages to the terminal after having sucessfully exited the python process with Ctrl-C.

sed: couldn't write 13 items to stdout: Broken pipe
uniq: write error: Broken pipe
uniq: write error: Broken pipe
sed: couldn't flush stdout: Broken pipe
sed: couldn't flush stdout: Broken pipe
sed: couldn't write 1 item to stdout: Broken pipe

Is there any way to avoid output like the above using joblib Parallel?

edited Sep 06 '15 at 18:06

John Kugelman

349,597
67
533
578

asked Sep 06 '15 at 17:31

The Unfun Cat

29,987
31
114
156

ctrl+c is used to kill a process whether it is complete or in-process.By using ctrl+c you terminate the program therefore you get Broken pipe message. – Golden_flash Sep 06 '15 at 17:36
How do I avoid this? Thanks for the info btw – The Unfun Cat Sep 06 '15 at 17:38
Can you add the code you are writing in the terminal ?It might be helpful for better understanding. – Golden_flash Sep 06 '15 at 17:41
Try closing `sys.stdin` in the python process. – Colonel Thirty Two Sep 06 '15 at 18:09
@TheUnfunCat To control the behaviour of your program when you press CTRL+C, this answer may help you : http://stackoverflow.com/a/1112363/740316 – Sep 06 '15 at 18:50
Are those handlers sent down to the started parallel processes? I doubt it... – The Unfun Cat Sep 07 '15 at 06:20

If starting unix pipe with Parallel command, pipe commands complain after cntrl-c

0 Answers0