Java concurrent writes from multiple threads to a single text file?

Question

I have a multi-threaded Java 7 program (a jar file) which uses JDBC to perform work (it uses a fixed thread pool).

The program works fine and it logs things as it progresses to the command shell console window (System.out.printf()) from multiple concurrent threads.

In addition to the console output I also need to add the ability for this program to write to a single plain ASCII text log file - from multiple threads.

The volume of output is low, the file will be relatively small as its a log file, not a data file.

Can you please suggest a good and relatively simple design/approach to get this done using Java 7 features (I dont have Java 8 yet)?

Any code samples would also be appreciated.

thank you very much

EDIT:

I forgot to add: in Java 7 using Files.newOutputStream() static factory method is stated to be thread safe - according to official Java documentation. Is this the simplest option to write a single shared text log file from multiple threads?

Why not go for a standard logging API like slf4j http://www.slf4j.org/manual.html? — Nitin Dandriyal, Jun 03 '15 at 06:29
Or else if you want to stick to only jdk api see this: http://stackoverflow.com/questions/5950557/good-examples-using-java-util-logging — Nitin Dandriyal, Jun 03 '15 at 06:34
forgot to add: in Java 7 using Files.newOutputStream() static factory method is stated to be thread safe - according to official Java documentation. Is this the best option to write a single shared text log file from multiple threads? — Acid Rider, Jun 03 '15 at 06:44

llogiq · Answer 1 · 2015-06-03T07:14:17.087

4

If you want to log output, why not use a logging library, like e.g. log4j2? This will allow you to tailor your log to your specific needs, and can log without synchronizing your threads on stdout (you know that running System.out.print involves locking on System.out?)

Edit: For the latter, if the things you log are thread-safe, and you are OK with adding LMAX' disruptor.jar to your build, you can configure async loggers (just add "async") that will have a logging thread take care of the whole message formatting and writing (and keeping your log messages in order) while allowing your threads to run on without a hitch.

edited Jun 03 '15 at 07:14

answered Jun 03 '15 at 06:24

llogiq

13,815
8
40
72

Java has logger included in JDK (java.util.Logger). In documentation they have mentioned that All methods on Logger are multi-thread safe. https://docs.oracle.com/javase/7/docs/api/java/util/logging/Logger.html – Sagar Gandhi Jun 03 '15 at 07:21
3

Yes, but the default logger implementation is dog slow when compared to log4j2. Granted, it's probably still faster than System.out.println, but why not use the currently best option? By the way, the *things* you log must be thread-safe, because e.g. if you log a Map and it is modified in the meantime, your logger may throw ConcurrentModificationException. In that case, either copy or use .toString() within the log. call. – llogiq Jun 03 '15 at 07:23

score 2 · Answer 2 · answered Jun 03 '15 at 06:21

Given that you've said the volume of output is low, the simplest option would probably be to just write a thread-safe writer which uses synchronization to make sure that only one thread can actually write to the file at a time.

If you don't want threads to block each other, you could have a single thread dedicated to the writing, using a BlockingQueue - threads add write jobs (in whatever form they need to - probably just as strings) to the queue, and the single thread takes the values off the queue and writes them to the file.

Either way, it would be worth abstracting out the details behind a class dedicated for this purpose (ideally implementing an interface for testability and flexibility reasons). That way you can change the actual underlying implementation later on - for example, starting off with the synchronized approach and moving to the producer/consumer queue later if you need to.

thanks Jon, I implemented the Queue and a separate thread to write to the file, seems to work well. — Acid Rider, Jun 04 '15 at 06:55

score 0 · Answer 3 · answered Jun 03 '15 at 06:20

0

Keep a common PrintStream reference where you'll write to (instead of System.out) and set it to System.out or channel it through to a FileOutputStream depending on what you want.

Your code won't change much (barely at all) and PrintStream is already synchronized too.

answered Jun 03 '15 at 06:20

Kayaman

72,141
5
83
121

1

On the other hand, `PrintStream` swallows exceptions - and I can't see anything in the *documentation* saying operations are synchronized... – Jon Skeet Jun 03 '15 at 06:22
Well, I was considering suggesting what you answered, but went for the extreme low effort version. – Kayaman Jun 03 '15 at 06:25
@Ouney Yes, we know that the Hotspot version of `PrintStream` is thread-safe. However it's not asserted in any documentation (and we don't know if OP is using Hotspot). – Kayaman Jun 03 '15 at 06:30

Java concurrent writes from multiple threads to a single text file?

3 Answers3