1

I am trying to write a json file using this code:

        File f = new File("words_3.json");
        if (!f.exists()) {
            f.createNewFile();
        }
        if (fileWriter == null)
            fileWriter = new BufferedWriter(new FileWriter(f));
        while (scanner.hasNext()) {
            String text = scanner.nextLine(); 
                    fileWriter.append(text);
                    System.out.println("writing : "+text);
        }

Statement System.out.println() shows all text in the terminal.

When I'm checking the output file, I see that only 1300 lines has been written, while there are more than 2000 lines available.

Alexander Ivanchenko
  • 25,667
  • 5
  • 22
  • 46
eFortsHub
  • 28
  • 5

1 Answers1

2

The data that you're writing in to an output stream isn't guaranteed to reach its destination immediately.

The BufferedWritter is a so-called high-level stream which decorates the underlying stream that deals with a particular destination of data like FileWriter (and there could be a few more streams in between them) by buffering the text output and providing a convince-method newLine().

BufferedWritter maintains a buffer (an array of characters) with a default size of 8192. And when it gets full, it hands it out to the underlying low-level stream. In this case, to a FileWriter, which will take care of encoding the characters into bytes.

When it's done, the JVM will hand the data out to the operating system via FileOutputStream (because under the hood character streams are build on top of bite streams).

So, the data written to the buffer will appear in a file in chunks:

  • when the buffer gets full;
  • and after the stream was closed.

Javadoc for method close() says:

Closes the stream, flushing it first.

I.e. before releasing the resource close() invokes method flush() which forces the cached data to be passed into its destination.

If no exception occur, everything that was written into the stream is guaranteed to reach the destination when the stream is being closed.

You can also use flush() in your code. But it has to applied with great caution. Probably when you deal with large amounts of critical data and which is useful, even when partially written (so in case of exceptions you'll lose less information). Misusing the flush() could significantly reduce the performance.

Alexander Ivanchenko
  • 25,667
  • 5
  • 22
  • 46
  • _"The JVM hands it out to the operating system, which will store this data in the cache."_ this is not correct, a buffered writer has a *buffer* inside Java. – Mark Rotteveel Mar 23 '22 at 16:50
  • @MarkRotteveel I know that. So when `BufferedWriter` will pass its *8 KB* of data to the system, it'll be written as soon as possible? Or it depends on something? – Alexander Ivanchenko Mar 23 '22 at 16:56
  • 2
    The actual write to disk might be delayed by the OS, but in general, when another process tries to read the file, the OS cache will flush (or the read will be satisfied from the cache), so the problem here is wholly inside the JVM, and dragging the OS into it is an unnecessary complication. – Mark Rotteveel Mar 23 '22 at 16:59
  • @MarkRotteveel Thanks for explanation. I've amended the answer. – Alexander Ivanchenko Mar 23 '22 at 17:11
  • Duplicates are only automatically removed if the question has no answers. Sometimes people will vote to delete a duplicate, but this is not what should normally done (duplicates serve as sign posts to the other question). However, if you want you can delete your answer here and post it on the duplicate. – Mark Rotteveel Mar 23 '22 at 19:29
  • 1
    When the `BufferedWriter`’s buffer is full, it just hands the data over to the underlying `Writer`, which is still a Java class. In case of `FileWriter`, it has to encode the characters to bytes, which requires a `ByteBuffer`. Only when this buffer is full (or flush/close happens), the actual bytes are handed over to the `FileOutputStream` which will call into the operating system). Since the encoded 8192 chars may need more than 8192 bytes, it’s possible that some of them are written and the remaining get buffered in `FileWriter`’s byte buffer when `BufferedWriter`’s buffer is full. – Holger Mar 24 '22 at 09:48
  • @Holger Thanks for point out on these details, I've updated the answer. – Alexander Ivanchenko Mar 25 '22 at 13:51