I would like to know if Log4j2 has been built with that in mind or if things may (crash | lose logs | etc) at very high concurrency

Question

This question has been askedfor log4j but not log4j2: Is it safe to use the same log file by two different appenders

Technically, you can create multiple appenders in Log4j2 that write in the same file. This seems to work well.

Here's my OS / JDK :

• Oracle JDK 7u45
• Ubuntu LTE 14.04

Here's a sample configuration (in yaml) :

Configuration:
  status: debug

  Appenders:
    RandomAccessFile:
      - name: TestA
        fileName: logs/TEST.log
        PatternLayout:
          Pattern: "%msg%n"
      - name: TestB
        fileName: logs/TEST.log
        PatternLayout:
          Pattern: "%msg%n"

  Loggers:
    Root:
      level: trace
      AppenderRef:
        - ref: TestA
        - ref: TestB

My Java sample :

final Logger root = LoggerFactory.getLogger(Logger.ROOT_LOGGER_NAME);
root.trace("!!! Trace World !!!");
root.debug("!!! Debug World !!!");
root.info("!!! Info World  !!!");
root.warn("!!! Warn World  !!!");
root.error("!!! Error World !!!");

My log file result :

20150513T112956,819 TRACE "!!! Trace World !!!"
20150513T112956,819 TRACE "!!! Trace World !!!"
20150513T112956,819 DEBUG "!!! Debug World !!!"
20150513T112956,819 DEBUG "!!! Debug World !!!"
20150513T112956,819 INFO  "!!! Info World  !!!"
20150513T112956,819 INFO  "!!! Info World  !!!"
20150513T112956,819 WARN  "!!! Warn World  !!!"
20150513T112956,819 WARN  "!!! Warn World  !!!"
20150513T112956,819 ERROR "!!! Error World !!!"
20150513T112956,819 ERROR "!!! Error World !!!"

I would like to know if Log4j2 has been built with that in mind or if things may (crash | lose logs | etc) at very high concurrency.

UPDATE :

I ran this Benchmark test and there is no missing log. Still, I'm not sure this test fully solves the question. :

public class Benchmark {

    private static final int nbThreads = 32;
    private static final int iterations = 10000;
    static List<BenchmarkThread> benchmarkThreadList = new ArrayList<>(nbThreads);
    private static Logger root;

    static {
        System.setProperty("Log4jContextSelector", "org.apache.logging.log4j.core.async.AsyncLoggerContextSelector");
    }

    public static void main(String[] args) throws InterruptedException {

        root = LoggerFactory.getLogger(Logger.ROOT_LOGGER_NAME);


        // Create BenchmarkThreads
        for (int i = 1; i <= nbThreads; i++) {
            benchmarkThreadList.add(new BenchmarkThread("T" + i, iterations));
        }

        root.error("-----------------------------------------------------------");
        root.error("----------------------  WARMUP  ---------------------------");
        root.error("-----------------------------------------------------------");

        // Warmup loggers
        doBenchmark("WARMUP", 100);

        Thread.sleep(100);
        root.error("-----------------------------------------------------------");
        root.error("---------------------  BENCHMARK  -------------------------");
        root.error("-----------------------------------------------------------");

        // Execute Benchmark
        for (int i = 0; i < nbThreads; i++) {
            benchmarkThreadList.get(i).start();
        }

        Thread.sleep(100);
        root.error("-----------------------------------------------------------");
        root.error("----------------------  FINISHED  -------------------------");
        root.error("-----------------------------------------------------------");

    }

    protected static void doBenchmark(String name, int iteration) {
        for (int i = 1; i <= iteration; i++) {
            root.error("{};{}", name, i);
        }
    }


    protected static class BenchmarkThread extends Thread {

        protected final int iteration;
        protected final String name;

        public BenchmarkThread(String name, int iteration) {
            this.name = name;
            this.iteration = iteration;
        }

        @Override
        public void run() {
            Benchmark.doBenchmark(name, iteration);
        }
    }

}

Answer 1

UPDATE: I did not realize that you are already using Async Loggers. In that case this is indeed a log4j2 question. :-)

The answer is yes, log4j2 and especially Async Loggers are designed with very high concurrency in mind. Multiple loggers in multiple threads can log concurrently, and the resulting log messages are put on a lock-free queue for later processing by the background thread. There is a single background thread that calls all appenders sequentially, so even if multiple appenders write to the same file, no messages are dropped and messages from each logger thread are written fully before the next message is written (no partial writes).

In case of a crash, messages that were in the queue but have not been flushed to disk yet may be lost. This is a trade-off for performance and is the case with all asynchronous logging.

If you are logging synchronously (e.g. without using Async Loggers) it becomes a question of what file I/O atomicity guarantees the JVM and OS make.

PREVIOUS ANSWER: This is more of a JVM/OS question than a log4j2 question. Rephrased: if multiple threads concurrently write to the same file, will the resulting file contain all messages (nothing is lost, and all messages are complete and correct)? (You may want to specify your JVM vendor and version and OS name and version.)

If you are looking for a safe log4j2 configuration, consider using Async Loggers. With Async Loggers all appenders are called sequentially in the single shared background thread, so you are sure no corruption will occur. In addition you get nice performance benefits.