Atomic append string to file in Windows using Java

Question

According to this question, small file appends on Windows are atomic.

I am trying to leverage this in Scala / Java to avoid locking when appending a small int to a file.

I am finding that writes using FileOutputStream(..., true) from multiple threads are being interleaved, even though the question linked above suggests that they should be atomic.

A test harness for this in Scala can be found on my github

The key code is:

def invoked(id: Int, path: String) = {
  val writer = new FileOutputStream(path, true)
  val bytes = (id.toString + ';').getBytes(Charset.defaultCharset())
  writer.write(bytes)
  writer.close()
}

... I was hoping that "invoked" would be thread-safe without locking.

The same Java/Scala code achieves atomic file appends on Linux (as per this question small file appends are atomic on POSIX), so the difference seems to be in the native implementations of FileOutputStream.

Perhaps FileOutputStream is not passing the correct flags to Windows? Does anyone know how to make this work? Will I have to write a JNI DLL to get this to work, or is there a way I can do this with Java's standard libraries?

I think that Windows and POSIX both offer this guarantee, which is enough OSes for me. See http://stackoverflow.com/questions/1154446 — Rich, Mar 11 '14 at 13:15
The safest way to do what you want to do is to have your producer threads "write" to a single consumer thread. The single consumer thread handles the actual file writing. — Gilbert Le Blanc, Mar 12 '14 at 11:36
@GilbertLeBlanc that would be a decent approach, yes. However, if the O/S provides atomic file writes (and it does claim to), then it would be more efficient to take advantage of that. I have already implemented a more conventional workaround. This question is about how to make the atomic file writes approach work in Java. — Rich, Mar 18 '14 at 10:03

score 1 · Accepted Answer · answered Jul 07 '14 at 21:16

Write operation behavior depends on file opening flags. On Windows you need to use FILE_APPEND_DATA flag or use the special constants to achieve append. Posix require O_APPEND flag. Although Java has bool value for append parameter looking into its source code following comment explains why it does not work.

FileDispatcherImpl(boolean append) {
    /* append is ignored */
}

Atomic append string to file in Windows using Java

1 Answers1

Linked