5

There are a lot of ways to set output buffering off in Python: Disable output buffering

What makes me curious is how do I know that output buffering is really really already off? What is the best and simple way to check it?

Community
  • 1
  • 1
Aminah Nuraini
  • 18,120
  • 8
  • 90
  • 108
  • That really is interesting. I doubt it is at all possible though, because `sys` is C module, and you can not do much beyond what the interface provides you. – Dmitry Torba Sep 01 '16 at 19:31
  • Maybe there is something simpler? What is the clear difference we'd get if it's turned on or off? If there is no difference, we won't care about it right? It might need a GIF to explain. I don't even think an image is enough. – Aminah Nuraini Sep 01 '16 at 19:34
  • 1
    There is no need to care, really. As long as you know when it needs to be flushed, you can just call `sys.stdout.flush()` and that will work fine whether buffering is on or off. There are some tasks where it might be critical to flush on time, for example look at my answer to http://stackoverflow.com/questions/39255498/python-cgi-os-system-causing-malformed-header , but it is not like you need to know if data is buffered or not. That is just interesting thing to find out, but I think in that case you can run an experiment. – Dmitry Torba Sep 01 '16 at 19:41
  • I know. It's for educational purpose since a test I am going to take emphasize the importance of output buffering – Aminah Nuraini Sep 01 '16 at 19:49

3 Answers3

2

Try this: $ python myscript.py | cat

If it's unbuffered, output from your script will immediately display on your terminal. Otherwise it will be buffered by cat until a flush happens, either triggered by your script or when it finishes.

Willy Lee
  • 121
  • 1
  • 5
2

I've tested the stdout buffering on Linux with Python 3.8, and found that Python3's output buffer can detect if it is being run at a terminal (a "TTY"), in which case the output from print functions is displayed immediately, otherwise the output is delayed until the buffer is full or the script exits. It will decide that it's not at a TTY if the stdout is piped, for example. (Sorry, I've no idea what happens on Windows.)

This output delay can be prevented with the "flush=True" print argument, of course. It can also be prevented by disabling buffering with Python3's "-u" command-line argument, or by setting the shell environment variable PYTHONUNBUFFERED=true, in which case the buffer is replaced with direct FileIO.

All this can be detected within Python3 by querying the sys.stdout.buffer like so:

#!/usr/bin/env python3
import sys
print("Name: ", type(sys.stdout.buffer).__name__)
print("TTY? ", sys.stdout.buffer.isatty())

This script's output under the various conditions is:

$ myscript.py # Displays immediately
Name:  BufferedWriter
TTY?  True

$ myscript.py | tee # Waits for script exit because of pipe
Name:  BufferedWriter
TTY?  False

$ python3 -u myscript.py # Displays immediately
Name:  FileIO
TTY?  True

$ python3 -u myscript.py | tee # Also displays immediately despite pipe
Name:  FileIO
TTY?  False

Edit: And the buffering can also be disabled with the following shebang (assuming the script name is not prefixed with "python3" in the shell command.)

#!/usr/bin/env -S python3 -u
Dave Rove
  • 913
  • 1
  • 12
  • 18
1

After some experimentation I found this method to distinguish if python is buffering on stdout:

import sys

def is_stdout_buffered():
    # Print a single space + carriage return but no new-line (should have no visible effect)
    print " \r",
    # If the file position is a positive integer then stdout is buffered
    try:
        pos = sys.stdout.tell()
        if pos > 0:
            return True
    except IOError:  # In some terminals tell() throws IOError if stdout is unbuffered
        pass
    return False

I have tested it only on Windows in CMD, MinGW and Git Bash. The key test is Git Bash which defaults to buffered, unless you call it with python -u in which case it is unbuffered (Cmd and Git bash have a different result when run a Python code)

This check can be used at the start of terminal scripts to warn the user if they may be affected by delayed output e.g. from print statements if there is no flush.

sparrowt
  • 2,641
  • 25
  • 24