71

Is there any way to get the effect of running python -u from within my code? Failing that, can my program check if it is running in -u mode and exit with an error message if not? This is on Linux (Ubuntu 8.10 Server).

Henry Ecker
  • 34,399
  • 18
  • 41
  • 57
Martin DeMello
  • 11,876
  • 7
  • 49
  • 64
  • 6
    Another workaround is set [`PYTHONUNBUFFERED` env var](https://docs.python.org/3/using/cmdline.html#envvar-PYTHONUNBUFFERED) to any non-empty string. This is NOT in your code — must be set before python interpreter starts — but is easier than modifying whatever launches your script to add `-u`. – Beni Cherniavsky-Paskin May 11 '20 at 05:53
  • 1
    Perfect, thank you! This helped me immensely today. I was starting a FastAPI service from a bash file (running uvicorn), which is itself started from a SystemD .service file. I realized that some of my stdout was being buffered and only spilling out when the program terminated, not at the beginning when I wanted to see it. Since the Python startup is a couple of layers deep, I was trying to figure out how to convince uvicorn to add the "-u" when it started Python, but with this method, I was able to simply add a Environment=PYTHONUNBUFFERED=anystringhere to the service definition! Nice! – Jeff Saxe Sep 02 '22 at 16:00

4 Answers4

53

The best I could come up with:

>>> import os
>>> import sys
>>> unbuffered = os.fdopen(sys.stdout.fileno(), 'w', 0)
>>> unbuffered.write('test')
test>>> 
>>> sys.stdout = unbuffered
>>> print 'test'
test

Tested on GNU/Linux. It seems it should work on Windows too. If I knew how to reopen sys.stdout, it would be much easier:

sys.stdout = open('???', 'w', 0)

References:
http://docs.python.org/library/stdtypes.html#file-objects
http://docs.python.org/library/functions.html#open
http://docs.python.org/library/os.html#file-object-creation

[Edit]

Note that it would be probably better to close sys.stdout before overwriting it.

Bastien Léonard
  • 60,478
  • 20
  • 78
  • 95
  • 1
    `stdout` still lives in `sys.__stdout__` – Thomas Ahle Feb 28 '14 at 10:35
  • 22
    Note that this doesn't work in Py3K. `ValueError: can't have unbuffered text I/O` – vbo Sep 12 '14 at 14:27
  • 1
    The answers on this question are terribly out of date. Please refer to the linked question for Python 3.3+ answers: https://stackoverflow.com/q/107705/2111778 – xjcl Nov 21 '22 at 04:04
39

You could always pass the -u parameter in the shebang line:

#!/usr/bin/python -u
mikewaters
  • 3,668
  • 3
  • 28
  • 22
  • 25
    See title: 1. OP knows about it; 2. wants to do it programmatically. – Tobu Jan 23 '11 at 01:32
  • 21
    The author implies command-line usage, not indicating familiarity with the alternate invocation offered by the shell (shebang). Thanks for your helpful feedback though. – mikewaters Jan 25 '11 at 07:31
  • 14
    This doesn't seem to work with the "env trick". `#!/usr/bin/env python -u`. I get the following error `/usr/bin/env: python -u: No such file or directory`. If I remove the `-u` it works again. – Aaron McDaid Sep 25 '14 at 08:44
  • To follow up my comment a moment ago, I think this explains my problem: http://stackoverflow.com/questions/4303128/how-to-use-multiple-arguments-with-a-shebang-i-e – Aaron McDaid Sep 25 '14 at 08:47
  • 2
    (me again!). Here is a solution to that problem I discussed in my two comments above: http://stackoverflow.com/questions/17458528/why-does-this-snippet-work . (But yes, this isn't related to the original question here) – Aaron McDaid Sep 26 '14 at 09:51
  • This is neat... – dentex Oct 01 '16 at 17:11
  • 1
    As Aaron says, plainly adding the option to the shebang doesn't work. Instead, pass the `-S` option to the `env` command, like so: `#!/usr/bin/env -S python3 -u` – Nico Villanueva Oct 09 '20 at 14:10
9

Assuming you're on Windows:

msvcrt.setmode(sys.stdout.fileno(), os.O_BINARY)

... and on Unix:

fl = fcntl.fcntl(sys.stdout.fileno(), fcntl.F_GETFL)
fl |= os.O_SYNC
fcntl.fcntl(sys.stdout.fileno(), fcntl.F_SETFL, fl)

(Unix copied in from commented solution, rather than linking.)

Erik Aronesty
  • 11,620
  • 5
  • 64
  • 44
RichieHindle
  • 272,464
  • 47
  • 358
  • 399
  • sorry, forgot to add that. on linux (ubuntu) – Martin DeMello May 19 '09 at 10:27
  • 5
    @Martin DeMello: Please do not add new facts in comments. Please update your question with new facts. New facts in comments are hard to find. – S.Lott May 19 '09 at 10:37
  • 1
    Supported on unix, linux, and anything posix: http://stackoverflow.com/questions/107705/python-output-buffering/1736047#1736047 – Tobu Jan 23 '11 at 01:41
  • Note that this does not work with python2 (no error, output is still buffered). Found out when running a script with the wrong interpreter – Romuald Brunet Mar 31 '21 at 14:22
8

EDIT (Oct 2020). As pointed out in a note to this answer, in Python3, stderr is buffered too.

You might use the fact that stderr is never buffered and try to redirect stdout to stderr:

import sys
#buffered output is here
doStuff()

oldStdout = sys.stdout
sys.stdout = sys.stderr
#unbuffered output from here on

doMoreStuff()
sys.stdout = oldStdout

#the output is buffered again
doEvenMoreStuff()
Boris Gorelik
  • 29,945
  • 39
  • 128
  • 170