41

I have come across these two terms and my understanding of them seem to overlap with each other. Flush is used with buffers and sync is used to talk about persisting changes of file to disk.

In C, fflush(stdin) makes sure that the buffer is cleared. And fsync to persist changes file to disk.

If these concepts are not universally defined, would prefer a linux, java explanation.

I found a related post, but ir doesn't really answer my question. Really force file sync/flush in Java

Community
  • 1
  • 1
smartnut007
  • 6,324
  • 6
  • 45
  • 52
  • Specifically, I came across this book that relates to Java that says, "make sure to flush and sync". what does each of these steps involve ? I would like an answer that distinguishes both and also a scenario where both are involved. – smartnut007 Nov 01 '10 at 20:50

1 Answers1

84

In Java, the flush() method is used in output streams and writers to ensure that buffered data is written out. However, according to the Javadocs:

If the intended destination of this stream is an abstraction provided by the underlying operating system, for example a file, then flushing the stream guarantees only that bytes previously written to the stream are passed to the operating system for writing; it does not guarantee that they are actually written to a physical device such as a disk drive.

On the other hand, FileDescriptor.sync() can be used to ensure that data buffered by the OS is written to the physical device (disk). This is the same as the sync call in Linux / POSIX.

If your Java application really needs to ensure that data is physically written to disk, you may need to flush and sync, e.g.:

FileOutputStream out = new FileOutputStream(filename);

[...]

out.flush();
out.getFD().sync();

References:

Grodriguez
  • 21,501
  • 10
  • 63
  • 107
  • 12
    If I may restate this crudely. So, flush clears the jvm buffer and transfers it to the OS buffer. And sync makes sure the OS actually persists the contents to file. does that makes sense ? – smartnut007 Nov 01 '10 at 20:54
  • flush says "write out buffers". I am like write out where ? write out sounds synonymous to persist to me. Hence, confusing. – smartnut007 Nov 01 '10 at 20:58
  • "Write out" in this context means send the data to the actual destination of the output stream or writer. For file-based streams this typically means that any data that is internally buffered by the stream will be sent out to the OS. – Grodriguez Nov 01 '10 at 21:03
  • 1
    Perhaps it helps if you consider that internally, `FileOutputStream.flush()` may end up calling the POSIX function `fwrite()` to send any buffered data to the OS. – Grodriguez Nov 01 '10 at 21:06
  • The current FileOutputStream.flush() implementation is a NOOP - i.e. doesn't do anything. – Mark Mar 25 '14 at 05:37
  • @MarkCarter: That is an implementation detail and you should not rely on it. It may change in the future, or it may be implemented in a different way on a different platform. – Grodriguez Mar 25 '14 at 14:42
  • 1
    @Grodriguez: I'm not so sure. The FileOutputStream javadoc clearly shows that the method is not overridden and the superclass method javadoc states that the implementation does nothing. So, this is not merely an implementation detail. – Mark Mar 25 '14 at 15:06
  • @MarkCarter The fact that it is not overriden *is* an implementation detail. Unfortunately Javadocs sometimes give you too much information. – Grodriguez Mar 25 '14 at 15:10
  • @Grodriguez: The API is clear. It says that the implementation does nothing. There is no argument here! – Mark Mar 26 '14 at 00:47
  • 1
    @MarkCarter Indeed there's no argument here. The API says exactly this: "The general contract of flush is that calling it is an indication that, if any bytes previously written have been buffered by the implementation of the output stream, such bytes should immediately be written to their intended destination." Then goes on to say that for OutputStream, the default implementation does nothing. This last bit is intended for *developers* that subclass OutputStream. *Users* of the API should adhere to the general contract of the method. But of course you are free to write your code as you see fit – Grodriguez Mar 26 '14 at 07:44
  • 1
    I've never heard of an API that specifies some things for "devs" and other things for "users" without explicitly or implicitly saying that. The guesswork is too great. If a subclass wants to specify something different to the superclass then it will override that method even if only for the sake of documenting the fact in the javadoc. I would imagine very few users call FileOutputStream.flush(). For this reason, I would imagine it will remain a NOOP for a long long time. At the very least, FileOutputStream.flush() currently being a NOOP is a worthy addition to your otherwise excellent answer. – Mark Mar 26 '14 at 09:57
  • 1
    Mark: I don't agree with this view. The documentation for `OutputStream.flush()` precisely describes the **general contract** for the `flush` method. Programmers using output streams should adhere to this general contract. Skipping calls to `flush` because it happens to be a NOOP in a particular type of output streams is in my opinion a dangerous practice, and it does not actually provide any real benefits. – Grodriguez Mar 26 '14 at 17:30
  • Not to mention that the Javadocs for `OutputStream.flush()` even include an example of streams that write to a *file* in order to illustrate what `flush` is supposed to do... – Grodriguez Mar 26 '14 at 17:31
  • a related question on this: if I only call "out.flush()" and the process dies, will i be able to read the data that i write to if I start another process to read it? – yuyang Jun 26 '18 at 00:37
  • @Grodriguez As @mark pionted out earlier, in java 8, FileOutputStream class itself did not override the `flush` method. Hence it uses the implementation of `OutputStream.flush()` that does nothing. – yuyang Jul 02 '18 at 20:12
  • @Grodriguez in practice we need to deal with specific implementations. as far as the implementation of java 8 FileOutputStream.flush() is concerned, there is no guarantee that after calling flush(), the other processes can read the written data, or the OS will cache the data if the process crashes after calling FileOutputStream.flush() – yuyang Jul 03 '18 at 07:10
  • @yuyang If you want to discuss the actual implementation of FileOutputStream in Java 8 I suggest that you post a separate question. – Grodriguez Jul 04 '18 at 07:35
  • Any idea how this compares with `getChannel().force(boolean)`? – Gray Oct 31 '18 at 21:16
  • How can I sync the entire filesystem with Java? Also how do I sync the deletion of a file to disk? – JohnyTex Feb 21 '22 at 08:35