Java Synchronisation Atomicity

Question

My single-threaded application writes to two memory mapped byte buffers (backed by two memory mapped files) Another Java application is reading from these files. I would like to make writing to both buffers atomic/transactional so application reading the files either sees both changes or none. Does Java synchronize atomicity help here if writer writes to both buffers inside a synchronised block? I’m a bit skeptical about using synchronised keyword to ensure atomicity for variables (buffers) backed by external storage (files).

Edit: I understand, synchronized block isn't supposed to work between JVMs. My question is more about one JVM atomically calling two putLong() on two MappedByteBuffer instance variables. By doing this, as far as this JVM is concerned, it has done the job to atomically write in two buffers. Now it's up to underlying OS's implementation of memory-mapped-file, how/when it takes those Longs from the buffers and writes to the host files. Whenever values from these buffers are read to be written to files, at least either both buffers would have been updated or none. Hopefully, this would help understand my question.

 MappedByteBuffer buffer1 = fileChannel1.map(READ_WRITE, 0, Integer.MAX_VALUE);
 MappedByteBuffer buffer2 = fileChannel2.map(READ_WRITE, 0, Integer.MAX_VALUE);

...

        synchronized (ATOMIC_LOCK) {
            buffer1.putLong(index, value1);
            buffer2.putLong(index, value2);
        }

Answer 1

I would like to make writing to both buffers atomic/transactional so application reading the files either sees both changes or none. Does Java synchronize atomicity help here if writer writes to both buffers inside a synchronised block?

No, not at all. The synchronized only works inside one JVM. It does not do any type of synchronization for filesystem operations - which is what you would need here.

As to "make writing to both buffers atomic/transactional" - I don't think this is possible. Any guarantee for atomicity would have to come from the file system, and most do not provide such guarantees. Some filesystems and operating systems have limited guarantees, but that will depend on the specific filesystem in use (and its configuration).

So you will have to find a different solution.

Some alternative solutions that come to mind:

• You could lock the file - that would block the second application from also locking it. Both programs would need to lock the file whenever they want to read or write, so both programs cannot interfere - probably a shared lock for reading, and an exclusive lock for writing. See this question about details: How can I lock a file using java (if possible)
• Use several files and renaming - this is what many server applications do. Basically, you write the data under a temporary file name, and rename the file once you are done. Renaming is (usually) atomic, so this should be safe. The receiving application may only read the finished files, usually by adopting some naming convention. The maildir email storage format for example uses such a system.
• The best solution probably is not to communicate using shared files, but use a dedicated service instead - maybe some type of database, or a messaging bus, such as Apache Kafka or RabbitMQ.

Answer 2

The usual atomicity systems in java (such as synchronized, volatile, the "Java Memory Model" section of the Java Language Specification, classes like AtomicInteger, and any guarantees about atomicity provided by the spec as written in the javadoc of concurrency-relevant types, such as ConcurrentHashMap) all cover atomicity within a single JVM.

You're working on multiple JVMs so none of those guarantees hold.

You have a few options.

The obvious option - filesystem

The filesystem is commonly used as vehicle for inter-process atomicity, and java fully supports doing this:

Path p = Paths.get("/path/to/some/lockfile");
Files.createFile(p);

The createFile API guarantees atomicity - either that call succeeds in which case the file did not exist before but exists now, and that act was atomic (i.e. if 2 processes simultaneously execute createFile on the same path, at most one could possibly succeed).

More generally any of the 'create a file' calls on the Files class can do this, e.g. also Files.move can do this, which is fantastic for the idea of creating a file with content atomically (the above code makes an empty file): Create a file somewhere with a different temporary name, then Files.move it into the right name, asking for atomicity.

To do this, you pass the ATOMIC option:

Path p = someTempFile;
Path target = someTarget;
// code to fill p. Use try-with!
Files.move(p, target, StandardCopyOption.ATOMIC_MOVE);

If you are using a file system that doesn't support atomic operations, you'll get an exception that states this.

The major problem with this is - files outlast your JVM. So if you create a lock file and then your JVM hardcrashes, the lock file remains. You can add a runtime hook (with Runtime.getRuntime().addShutdownHook(new Thread(() -> { try { deleteLockFile(); } catch (Exception ignore) { } }));). This still won't work if your system is suddenly powered off or your JVM core dumps.

This is a problem all lock file based systems struggle with; generally on boot you check for the existence of the file and make some remark to the user that perhaps they wish to clean it out and printing the path to it. You can get fancy and include your PID or a timestamp when creating the lock file (but, remember, ONLY file creation can be atomic, to create a file atomically with content, you make the file in some temporary location and Files.move with the ATOMIC_MOVE flag).

ports

You can open an internet server port; only one process can do this and this too is atomic. If the port is already open you can even chat with the 'other' JVM which could be useful. There are no particular ATOMIC flags you need to pass, just follow any tutorial on opening TCP/IP ports. You will have to pick some port number between 1024 and 40000 and hope nobody else also picked that number, though.

Use an external system for the act

For example, if both JVMs talk to a single database, database have tons of ways to do things atomically; for example, psql has advisory locks which are specifically designed for application locks.