Filter out messages matching two filters in Log4j2

Question

I have a Java application that I'm running that uses log4j2.xml to configure logging. I want to drop messages that contain a certain string in the message and also have a certain MDC key.

If I specify a filter on the MDC like this:

<ThreadContextMapFilter onMatch="DENY" onMismatch="NEUTRAL">
  <KeyValuePair key="someValue" value="foo"/>
</ThreadContextMapFilter>

it correctly drops all messages that have this key-value pair, but I need it to only drop these if a RegexFilter also matches. Is there any way to do this? I see there is CompositeFilter but that is not able to perform this kind of boolean operation for me as far as I can tell.

I do not have the ability to modify source code of this application, all I've got to work with is the log4j2.xml

In `CompositeFilter` when the first filter returns ACCEPT or DENAY it exits. So you cannot make AND filtering configuration. The only thing that comes to mind is to create a custom filter, but, from what you write, it is not a solution for you... — jbrond, Sep 10 '21 at 06:52

score 2 · Answer 1 · answered Sep 25 '21 at 04:50

There are two ways you could accomplish what you want. (1) Create a CompositeFilter as you mentioned in your question using both the RegexFilter and ThreadContextMapFilter. (2) Use a ScriptFilter.

Below is some sample code to illustrate how you can do both of these. First, a class to generate some log events:

package example;

import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.apache.logging.log4j.ThreadContext;

public class SomeClass {

    private static final Logger log = LogManager.getLogger();   
    
    public static void main(String[] args){
        ThreadContext.put("someValue", "foo");
        log.info("Here's a test of info!"); //should not be accepted
        log.info("A message that doesn't match the regex"); //should be accepted
        ThreadContext.put("someValue", "bar");
        log.info("Another test of info"); //regex matches, context does not. Accepted.
        log.info("Some other message"); //does not match regex, context does not match. Accepted.
    }
}

Now the log4j2.xml configuration file:

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN">
    <Appenders>
        <Console name="Console" target="SYSTEM_OUT">
            <PatternLayout pattern="%d{yyy-MM-dd} [%t] %-5level %logger{36} %X{someValue} - %msg%n" />
        </Console>

        <File name="File1" fileName="logs/File1.log" immediateFlush="false"
            append="false">
            <PatternLayout
                pattern="%d{yyy-MM-dd} [%t] %-5level %logger{36} %X{someValue} - %msg%n" />
            <Filters>
                <ThreadContextMapFilter onMatch="NEUTRAL" onMismatch="ACCEPT">
                  <KeyValuePair key="someValue" value="foo"/>
                </ThreadContextMapFilter>
                <RegexFilter regex=".* test .*" onMatch="DENY" onMismatch="NEUTRAL"/>
            </Filters>

        </File>
        <File name="File2" fileName="logs/File2.log" immediateFlush="false"
            append="false">
            <PatternLayout
                pattern="%d{yyy-MM-dd} [%t] %-5level %logger{36} %X{someValue} - %msg%n" />
            <ScriptFilter onMatch="ACCEPT" onMisMatch="DENY">
              <Script name="MyFilter" language="JavaScript"><![CDATA[
                  if(logEvent.getContextMap().get("someValue") == "foo" 
                     && logEvent.getMessage().getFormattedMessage().matches(".* test .*")){
                        false
                    }else{
                        true
                    }
              ]]></Script>
            </ScriptFilter>
        </File>
    </Appenders>

    <Loggers>
        <Root level="ALL">
            <AppenderRef ref="Console"  />
            <AppenderRef ref="File1"  />
            <AppenderRef ref="File2"  />
        </Root>
    </Loggers>
</Configuration>

Here is sample console output:

2021-09-24 [main] INFO  example.SomeClass foo - Here's a test of info!
2021-09-24 [main] INFO  example.SomeClass foo - A message that doesn't match the regex
2021-09-24 [main] INFO  example.SomeClass bar - Another test of info
2021-09-24 [main] INFO  example.SomeClass bar - Some other message

Notice that all of the messages are accepted since the console appender has no filters.

The logs written to File1 and File2 are identical, sample output follows:

2021-09-24 [main] INFO  example.SomeClass foo - A message that doesn't match the regex
2021-09-24 [main] INFO  example.SomeClass bar - Another test of info
2021-09-24 [main] INFO  example.SomeClass bar - Some other message

Notice the 1 message where the regex and context value match has been excluded as desired.

Filter out messages matching two filters in Log4j2

1 Answers1

Linked