Multi threaded single file writing in Java

Question

This questions seems common, but I am facing issues while multi-threads are writing into the same file(Excel). Here is my code:

public class XLWriter {

private XLWriter() {
}

private static class SingletonHelper {
    private static final XLWriter INSTANCE = new XLWriter();
}

public static synchronized XLWriter getInstance() {
    return SingletonHelper.INSTANCE;
}

public static synchronized void writeOutput(Map<String, String> d) {
    try {
        --- Write file
    } catch (Exception e) {
        SOP("Not able to write output to the output file.");
    }
}

public static void createWorkBook(String fileName, String sheetName)
        throws IOException {
    try {
        -- Create workbook
    } catch (WriteException e) {
        System.out.println("Could not create workbook" + e);
    }
}

I am using testng framework and 10 threads try to write into the same file. Many of the threads are not able to write into it and goes inside the Exception block... What is the better way to do this? Any code samples will greatly help me as I have very less time to finish this.. Thank you.

Does it work with a single thread? The `createWorkBook` method is not `synchronized`. Is that intended? Which of the two methods fail? — neutrino, Nov 11 '13 at 13:34
Create a queue into which each thread writes its data. Create a separate writer thread which reads the queue data and persists to disk. — rec, Nov 11 '13 at 13:47
Possible duplicate of [Threads and file writing](http://stackoverflow.com/questions/9972549/threads-and-file-writing) — Vadzim, Oct 16 '15 at 15:45

TwoThe · Accepted Answer · 2013-11-11T14:01:00.637

You do not need to synchronize pure read, so public static synchronized XLWriter getInstance() can go without synchronized without any error. You only need to synchronize writes, where more than one thread could potentially write/read the same data at the same time.

There are two things that can be done to solve your problem. The easy one is to create a special write function which is then the only one to be synchronized:

private void write(final File f, final Map<String, String> output) {
  synchronized(f) {
    // do the write
  }
}

Synchronizing on f is correct, because this is the shared resource that has to be exclusively accessed.

You do not need to change the createWorkBook function as long as you tell the file creation to not overwrite existing files, because the file-system itself is already thread-safe. Now you just need a look-up table for files which are already opened to get the appropriate file handle, because there must be only one File per file opened at any time. This can be solved using a ConcurrentHashMap like this:

public void getFileFor(String filename) {
  File newFile = File(filename);
  File inList = fileHashMap.putIfAbsent(newFile);
  if (inList != null) {
    return inList;
  } else {
    return newFile;
  }
}

Just make sure that you have a solution somewhere to close all files in the hashmap once you are done.

For further ideas: you can as well create a single write-thread with the only purpose to write stuff into that file. If this thread has a ConcurrentBlockingQueue, other threads can add their part into this queue instead of the file, then proceed with whatever they were doing, while the write-thread is the only one who has access to the file ever. Therefore no write issues can arise, and the thread safety is completely handles in the ConcurrentBlockingQueue.

Thank you so much for this detailed answer. Very informative. Million thanks. — Mike, Nov 11 '13 at 14:14

Multi threaded single file writing in Java

1 Answers1