63

Why do certain streams need to be flushed (FileOutputStream and streams from Sockets) while the standard output stream does not?

Every time someone uses the System.out PrintStream object, be it while calling println() or write(), they never flush the stream. However, other programmers habitually call flush() a PrintStream/PrintWriter with other streams.

I've recently asked this question to several programmers and some believe that there is some background handling in Java to auto-flush the System.out stream but I can't find any documentation on that.

Something like this makes me wonder if simply calling System.out.println() is platform independent as some systems may need you to flush the stream.

Mike Samuel
  • 118,113
  • 30
  • 216
  • 245
Mark Nguyen
  • 737
  • 1
  • 5
  • 7

4 Answers4

60

System.out is based around a PrintStream which by default flushes whenever a newline is written.

From the javadoc:

autoFlush - A boolean; if true, the output buffer will be flushed whenever a byte array is written, one of the println methods is invoked, or a newline character or byte ('\n') is written

So the println case you mention is explicitly handled, and the write case with a byte[] is also guaranteed to flush because it falls under "whenever a byte array is written".

If you replace System.out using System.setOut and don't use an autoflushing stream, then you will have to flush it like any other stream.

Library code probably shouldn't be using System.out directly, but if it does, then it should be careful to flush because a library user might override System.out to use a non flushing stream.

Any Java program that writes binary output to System.out should be careful to flush before exit because binary output often does not include a trailing newline.

Mike Samuel
  • 118,113
  • 30
  • 216
  • 245
  • 5
    only valid `if true`, but is it? I could not find any documentation about the value of autoFlush for System.out... – user85421 Aug 23 '11 at 19:53
  • 1
    @Carlos, It is up to the system that bootstraps the JVM and launches the main class what `System.out` and `System.err` are bound to and how they flush. When you use the `java` binary to launch a class `System.out` is initialized to something like `new PrintStream(new FileOutputStream(FileDescriptor.out), true, System.getProperty("file.encoding"))` but other JVMs differ. Obviously a JVM that embeds applets does something different. – Mike Samuel Aug 23 '11 at 20:46
  • @MikeSamuel, So does it mean that we still *need* to flush anyway since the autoflush behavior is not **guaranteed** by the specs? – Pacerier Jun 18 '14 at 11:10
  • @Pacerier, There's nothing in the `System.{out,err}` docs saying they must be autoflushing but most Java programs assume they are so few VMs are going to startup without that being the case, but flushing manually sounds like a good idea if you're delivering the program output or important logging or auditing telemetry via them. – Mike Samuel Jun 18 '14 at 11:58
  • 1
    Flushing implicitly with `'\n'` or explicitly with `.flush()` works for Windows `cmd` but not for the `mintty` (aka. MSYS2 Terminal) for Windows. – veganaiZe Feb 03 '19 at 17:57
  • @veganaiZe Do you need to write a `"\r\n"` for that? I.e. does writing `System.lineSeparator()` and then flushing work consistently? – Mike Samuel Feb 05 '19 at 19:32
  • @MikeSamuel I've tried that and every combo that I can think up. I think it's a bug in `mintty` for Windows. – veganaiZe Feb 12 '19 at 19:54
  • @MikeSamuel It's a known issue for ALL programs compiled outside of Cygwin/MSYS environment. (MinGW != MSYS) https://github.com/mintty/mintty/wiki/Tips#inputoutput-interaction-with-alien-programs – veganaiZe Feb 13 '19 at 17:45
  • @veganaiZe, nice. – Mike Samuel Feb 13 '19 at 20:41
6

From the PrintStream documentation:

Optionally, a PrintStream can be created so as to flush automatically; this means that the flush method is automatically invoked after a byte array is written, one of the println methods is invoked, or a newline character or byte ('\n') is written.

Although I don't see it mentioned explicitly in the documentation, it's my understanding that System.out will perform this auto-flushing.

David Z
  • 128,184
  • 27
  • 255
  • 279
6

When you can't wait for the item to be displayed, flush the stream.

When the JVM goes down, not flushing the stream risks the item being lost in the display buffer, which might make the sensible error message telling you why the JVM went down lost forever. That makes debugging much more difficult, as people then tend to say, "but it didn't get here, because it would have printed this".

Edwin Buck
  • 69,361
  • 7
  • 100
  • 138
  • Aren't error messages unbuffered? – user2153235 Apr 14 '23 at 14:26
  • 1
    The PrintStream that implements the `System.err` interface usually has an option to autoflush. If you do that, anything that is written to the interface triggers an additional call to `flush()`. `System.err` and `System.out` can be reset after the JVM initializes to capture output (that's how JUnit gets the output during a test run). But every JVM I know of initializes `System.err` with a `PrintStream` that autoflushes. I suppose you could also use this approach to configured `System.out` to autoflush. – Edwin Buck Apr 15 '23 at 19:04
  • Sorry, I *assumed* that the troubleshooting messages were being printed to the error stream. But I understand now that the response was about standard output generated. Of course, we could try what you suggested, to configure the standard output stream to autoflush. I'm just a Java dabbler, so it would take me some online research (not that it's what I'm trying to do right now). – user2153235 Apr 16 '23 at 19:26
  • @user2153235 Personally, I wouldn't reconfigure `System.out` to autoflush, it defeats a lot the performance gains made by not having it autoflush, and is sort of off-the-beaten-path of software assumptions. Instead, I would your error messages to go to `System.err` and use the flushing policy already applied to `System.err` which likely autoflushes (but if it doesn't, it is because someone / something else overrode it, typically for a reason). And no worries about the missteps, it's how one transitions from a dabbler to a skilled practitioner. – Edwin Buck Apr 16 '23 at 19:35
  • @user2153235 A good "simple approach" to this is to put a `try / catch / finally` block around the contents of your main method. In that, flush `System.out` in the `finally` block. This assumes that you never use `System.exit(...)`. Using `System.exit(...)` honestly causes more issues in program flow control than it solves, so avoiding it is a pretty good idea. – Edwin Buck Apr 16 '23 at 19:39
  • I'm never going to be more than a dabbler, because I'm not a developer. However, I do have to deal with Java code. I'm not trying to make standard output behave like an error stream, just trying to understand your answer. I was happened upon in during a much simpler task of trying to understand the buffering of the standard output stream. I appreciate your suggestion using the try/catch/finally block, which I will keep in mind for the future. Thanks! – user2153235 Apr 17 '23 at 22:54
  • @user2153235 No problem. The world is full of dabblers, and they eventually become experts. If you want an error stream, I would suggest using System.err instead of System.out. It auto flushes and was created (even before Java was created) to capture errors. Buffering output without flushing (like with System.out) is a good thing, as the cost of flushing output on each write is rather expensive in computer time (so your program will run slower, without doing more). The idea behind buffering is to collect multiple writes and then send them in a batch. – Edwin Buck Apr 19 '23 at 09:33
  • @user2153235 The problem with sending them in a batch is that when the program exits prior to sending the batch of items to be written, those buffered items to be written are lost. Why is it so expensive? Because transferring data from one program to another requires the operating system to do it on your program's behalf. So a flushed write writes to the buffer, then calls the OS to handle the write, then waits for that to finish. A non-flushed writes to the buffer, which is flushed when it is big / old enough, and doesn't block on the write. – Edwin Buck Apr 19 '23 at 09:36
  • I appreciate the background. I'm familiar with those concepts from unix's standard out and standard error streams. They also have similar counterparts in C++, but I haven't touched that in over a decade. Currently, I'm a Matlabber, which also has those concepts. The reason why I looked this up for Java was to streamline some code and get rid of flushing. Since the default is to autoflush on a newline, that is sufficiently fine-grain for me. – user2153235 Apr 19 '23 at 14:43
3

System.out is by default line-buffered. So if you are calling println and not print it should not be a problem. See this article for more info.

1ac0
  • 2,875
  • 3
  • 33
  • 47
Connor Doyle
  • 1,812
  • 14
  • 22