1

I'm implementing a logging for multiple threads, each will write into a List. After all threads have finished I will then dump the contents of the List into a file. Which concurrent List implementation should I use? I'm considering the ConcurrentLinkedQueue.

  • The writing will be concurrent, but the reading will be done by one thread, after all other threads have finished writing.

I could use a List for each thread but then I would have the overhead of managing multiple Lists and I'm not sure it is worth it. Another option would be a synchronized List.

Bonus question: How do I make the last thread dump the list into the file? See Multiple threads arrive, the last should do the processing.

Roland
  • 7,525
  • 13
  • 61
  • 124
  • ArrayBlockingQueue ? – StanislavL May 24 '17 at 06:59
  • @StanislavL that would require me to know the capacity in advance, I don't have this information. – Roland May 24 '17 at 07:03
  • Please provide a clear, concise, example of the problem. I can't read from your question what you are trying to achieve and what code problems you are running into. Maybe https://softwareengineering.stackexchange.com/ is a better place for this questio. – M. le Rutte May 24 '17 at 07:20

2 Answers2

1

Please - don't.

Java has a logging framework already built in, and it is fully customizable to create different handlers and formatters. If you don't like the standard Java logging framework for whatever reason there are still many others available. All these frameworks handle concurrency.

With the time you are spending on creating a framework that has already existed and perfected for a long time you could have created stuff that added something new to the world.

M. le Rutte
  • 3,525
  • 3
  • 18
  • 31
0

If you can in fact guarantee that any attempt to read the queue will happen "once all other threads have finished writing" then you do not need a concurrent queue at all.

All you need is a regular, run-of-the-mill, non-concurrent queue, and a factory of "log entry consumers" which dispenses a log entry consumer to each thread that needs to write to the queue.

The use of a consumer hides the queue, so the only thing that threads can do is append to the queue. The implementation of the consumer simply guarantees synchronized access to the insertion point of queue.

The queue then does not need to be synchronized because if it is only to be read "after all other threads have finished writing" then obviously, there will be no concurrent access happening while it is being read.

That having been said:

  • Are you sure about the "after all other threads have finished writing" part?

  • How do you define "all other threads have finished writing"?

  • How do you know that "all other threads have finished writing"?

  • Why is it important to you that the list is only read after "all other threads have finished writing"?

Mike Nakis
  • 56,297
  • 11
  • 110
  • 142
  • So synchronized List would be the answer. And you still need to synchronize the reader after the writers for memory visibility effects. To answer your questions: All writing Threads run in `CompletableFuture`s and I wait until they are finished("completed"). It is important that they are finished because after that the contents of the `List` get dumped into a file. – Roland May 24 '17 at 07:29
  • More than one thread writing to the same queue would still require a concurrent queue. – bowmore May 24 '17 at 09:16
  • @bowmore no, it would not. I explained that all you need is the consumers to guarantee synchronized access to the insertion point of the queue. A concurrent queue would introduce additional overhead when reading the queue, which is unnecessary, if the OP can somehow guarantee that the reading will only happen after all the writing is done. – Mike Nakis May 24 '17 at 09:19
  • I see you refer to the writers as 'consumers'. Usually these are referred to as the producers. – bowmore May 24 '17 at 09:21
  • @bowmore no, they are consumers. You have something you want to log, you give it to the consumer, you don't care what happens to it. The implementation of the consumer takes care of concurrency and puts the item in the (otherwise not concurrent) queue. – Mike Nakis May 24 '17 at 11:25
  • gotcha. Basically you're manually organizing synchronization for the write access to the queue. – bowmore May 24 '17 at 12:05
  • That's correct. A producer could be utilized on the other end of the queue, (the "reading" end, that "produces" items,) but there is no added value that it could bring to the picture. – Mike Nakis May 24 '17 at 12:08