Why do I need to use `flush=True` if printing when stdout isn't a TTY in Python but not in Java or Go?

Question

Given the following small program:

#!/usr/bin/env python3
import time

for i in range(15):
    print(f'{i}: sleeping')
    time.sleep(1)

When I run it with stdout attached to the terminal directly, I get the output pretty much immediately:

./sync_test.py

However, if I run my program with stdout attached to a pipe (which on the other end cat is listening and printing to the terminal), I don't get the output until the program ends:

./sync_test.py | cat

The only way I can get the output to be printed before the program quits is if I add flush=True to all of my print statements.

I tried this out in a few other languages (Go, Java, and C) and got mixed results. In Go and Java, the programs print immediately when stdout is not attached to a TTY. But with C, I experience the same behavior as I do with Python.

The Go program (test_go.go):

package main

import (
    "fmt"
    "time"
)

func main() {
    for i := 0; i < 15; i++ {
        fmt.Printf("%d: sleeping\n", i)
        time.Sleep(1 * time.Second)
    }
}

The Java program (test_java.java):

public class test_java {
    public static void main(String[] args) throws Exception{
        for(int i=0; i<15; i++){
            System.out.println(i + ": sleeping");
            Thread.sleep(1000);
        }
    }
}

The C program (test_c.c):

#include <stdio.h>
#include <unistd.h>
int main() {
    for(int i=0; i < 15; i ++) {
        printf("Hello, World!\n");
        sleep(1);
    }
    return 0;
}

(all programs were run with | cat at the end of the command)

I understand that stdout is buffered. What is the explanation for the differences in behavior across languages? And whether stdout is connected to a TTY or not? Are Java and Go flushing the buffers implicitly whereas C and Python are not?

Answer 1

The answer to "Why does Python do this?" is "To be roughly consistent with the C standard", where, by default, being connected to a tty makes stdout line buffered, and block buffered otherwise. Back in the Python 2 days, sys.stdout (and the file objects returned by regular open, but not by io.open) was actually built on top of C's stdio.h calls, so this behavior was inherited from C directly; in Python 3, stdio.h was removed from the picture (Python 3 uses its own buffering wrappers for I/O, which when flushed, call the underlying OS I/O routines directly, with no C-style FILE*s involved at all), but they preserved the behavior to minimize compatibility issues for existing code.

The reason C (and some other languages) make it line-buffered when connected to a tty is to allow the user to see complete lines immediately. Otherwise it's block-buffered because it's assumed no one is reading the lines as they appear anyway, so reducing system calls (by buffering more before each bulk write system call) is beneficial.

System calls aren't that expensive nowadays, so it's perfectly reasonable for a modern language to default to line buffering even when not connected to a tty; it'll run slower, but a few thousand cycles here and there isn't that much in the grand scheme of things. Go and Java aren't wrong to behave the way they do, and Python and C aren't wrong to go the other way. The behavior is consistent, and it works well enough for common cases, and it's trivial to add flushes (or otherwise mess with the default buffering, e.g. with the -u command line switch for Python, or the setvbuf function in C), and changing it might change the behavior of existing code, so there's no strong incentive to go back and alter the default behavior.