1

I am working on a cluster and using Python to do some calculations.

However, python scripts does not print anything until they finish or get killed.

I have checked some ways (from Disable output buffering)

  1. Use the -u command line switch
  2. Wrap sys.stdout in an object that flushes after every write
  3. Set PYTHONUNBUFFERED env var
  4. sys.stdout = os.fdopen(sys.stdout.fileno(), 'w', 0)

But none of these works.

The python version is 2.7.8, 2.7.3 and 2.4.2 (There are three versions on the system, which must be put into PATH before running)

Also, because the cluster uses NFS, I have the exactly the same python among multiple machines. But on one machine, it prints unbuffered (at least it prints something before ending), without any setting at all (no PYTHONUNBUFFERD). On other machines it does not, which makes me even more confusing.

My testing script is 

import os,sys,time
for i in range(100):
    print(i)
    time.sleep(1)

And

import os,sys,time
sys.stdout = os.fdopen(sys.stdout.fileno(),'w',0)
for i in range(100):
    print(i)
    time.sleep(1)

And

import os,sys,time
for i in range(100):
    print(i)
    sys.stdout.flush()
    time.sleep(1)

Is there any other way to make Python print unbuffered (or buffered more reasonably, like buffer for one line) without modifing Python codes (because there are a lot of them)? Thanks.

Community
  • 1
  • 1
Kala
  • 43
  • 6
  • That's weird. You shouldn't need to flush stdout. Your 1st script works fine here; I haven't bothered testing the others. What OS are you running on, and what shell are you running these scripts in? – PM 2Ring Dec 11 '14 at 07:58
  • http://stackoverflow.com/questions/12827256/python-standard-idiom-to-set-sys-stdout-buffer-to-zero-doesnt-work-with-unicode?rq=1 This might be of some help? – nchen24 Dec 11 '14 at 08:06
  • @PM2Ring CentOS 5.4 and bash. I have tried to do the same thing in bash and it prints something. However, I have also another computer at hand with the same system and works fine. – Kala Dec 12 '14 at 11:25
  • @nchen24 This still does not works. – Kala Dec 12 '14 at 11:35
  • Tricky. It's hard to debug problems that you can't reproduce. Are all the machines that do the unwanted buffering running the same OS & version of bash? Are they running GUI desktops or are they CLI-only systems? Can you reproduce the unwanted buffering with a simple bash or awk script? – PM 2Ring Dec 12 '14 at 12:03
  • Not all machines, there are at least one does not. They are CLI only. Only Python does not prints. And what is even weird is that it prints something today so I can not reproduce it – Kala Dec 15 '14 at 14:20

0 Answers0