0

I know this question comes up all the time but I am totally stumped.

I am trying to write a KafkaStreams application logging with Log4j2 and I am getting a Log4j (version 1) warning about no appender:

log4j:WARN No appenders could be found for logger (org.apache.commons.beanutils.converters.BooleanConverter).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

And the standard answer is to add an exclusion in the pom file which I have done and done again to no avail.

It gets even weirder because even though the warning might suggest common-beanutils the line that causes the error in the console output is:

        final Map<String, String> serdeConfig = Collections.singletonMap(
            AbstractKafkaAvroSerDeConfig.SCHEMA_REGISTRY_URL_CONFIG,
            ApplicationConfiguration.SCHEMA_REGISTRY_HOSTS);

I have spent an entire day on this and just don't seem to be making any progress at all.

After today my pom file is horribly cluttered with exclusions and my dependencies looks like this.

    <dependencies>

    <!--Configuration Dependency for importing properties file-->
    <dependency>
        <groupId>org.apache.commons</groupId>
        <artifactId>commons-configuration2</artifactId>
        <version>2.2</version>
        <!--Excluded in Map-->
        <exclusions>
            <exclusion>
                <artifactId>commons-logging</artifactId>
                <groupId>commons-logging</groupId>
            </exclusion>
        </exclusions>
    </dependency>

    <!--Dependency used for Apache commons configuration2-->
    <dependency>
        <groupId>commons-beanutils</groupId>
        <artifactId>commons-beanutils</artifactId>
        <version>1.9.3</version>
        <exclusions>
            <exclusion>
                <groupId>log4j</groupId>
                <artifactId>log4j</artifactId>
            </exclusion>
            <exclusion>
                <groupId>org.slf4j</groupId>
                <artifactId>slf4j-log4j12</artifactId>
            </exclusion>
            <!--Excluded in Map-->
            <exclusion>
                <artifactId>commons-logging</artifactId>
                <groupId>commons-logging</groupId>
            </exclusion>
        </exclusions>
    </dependency>

    <!--Log4j2 Dependency for logging-->
    <dependency>
        <groupId>org.apache.logging.log4j</groupId>
        <artifactId>log4j-api</artifactId>
        <version>2.11.0</version>
    </dependency>

    <!--Required for Log4j2 Dependency for logging-->
    <dependency>
        <groupId>org.apache.logging.log4j</groupId>
        <artifactId>log4j-core</artifactId>
        <version>2.11.0</version>
    </dependency>

    <!--Used for Object Json Serialization and Deserialization-->
    <dependency>
        <groupId>com.fasterxml.jackson.core</groupId>
        <artifactId>jackson-databind</artifactId>
        <version>2.9.6</version>
    </dependency>

    <!--Sumo Logic Dependency for Log4j2 logging-->
    <dependency>
        <groupId>org.apache.httpcomponents</groupId>
        <artifactId>httpclient</artifactId>
        <version>4.5.2</version>
        <exclusions>
            <exclusion>
                <groupId>log4j</groupId>
                <artifactId>log4j</artifactId>
            </exclusion>
            <exclusion>
                <groupId>org.slf4j</groupId>
                <artifactId>slf4j-log4j12</artifactId>
            </exclusion>
            <!--Excluded in Map-->
            <exclusion>
                <artifactId>commons-logging</artifactId>
                <groupId>commons-logging</groupId>
            </exclusion>
        </exclusions>
    </dependency>

    <!--Confluent Platform Dependency for streams avro serializer-->
    <dependency>
        <groupId>io.confluent</groupId>
        <artifactId>kafka-streams-avro-serde</artifactId>
        <version>3.3.0</version>
        <!--Required due to SLF4j dependency conflict-->
        <exclusions>
            <exclusion>
                <groupId>log4j</groupId>
                <artifactId>log4j</artifactId>
            </exclusion>
            <exclusion>
                <groupId>org.slf4j</groupId>
                <artifactId>slf4j-log4j12</artifactId>
            </exclusion>
            <!--Excluded in Map-->
            <exclusion>
                <artifactId>slf4j-api</artifactId>
                <groupId>org.slf4j</groupId>
            </exclusion>
        </exclusions>
    </dependency>

    <!--Confluent Platform Dependency for avro serializer-->
    <dependency>
        <groupId>io.confluent</groupId>
        <artifactId>kafka-avro-serializer</artifactId>
        <version>3.3.0</version>
        <!--Excluded in Map-->
        <exclusions>
            <exclusion>
                <groupId>log4j</groupId>
                <artifactId>log4j</artifactId>
            </exclusion>
            <exclusion>
                <groupId>org.slf4j</groupId>
                <artifactId>slf4j-log4j12</artifactId>
            </exclusion>
            <exclusion>
                <artifactId>commons-logging</artifactId>
                <groupId>commons-logging</groupId>
            </exclusion>
        </exclusions>
    </dependency>

    <!--Confluent Platform Dependency for schema registry-->
    <dependency>
        <groupId>io.confluent</groupId>
        <artifactId>kafka-schema-registry-client</artifactId>
        <version>3.3.0</version>
        <!--Required due to SLF4j dependency conflict-->
        <exclusions>
            <exclusion>
                <groupId>log4j</groupId>
                <artifactId>log4j</artifactId>
            </exclusion>
            <exclusion>
                <groupId>org.slf4j</groupId>
                <artifactId>slf4j-log4j12</artifactId>
            </exclusion>
        </exclusions>
    </dependency>

    <!--Kafka API Client Dependency-->
    <dependency>
        <groupId>org.apache.kafka</groupId>
        <artifactId>kafka-clients</artifactId>
        <version>1.1.0</version>
        <!--Excluded in Map-->
        <exclusions>
            <exclusion>
                <artifactId>slf4j-api</artifactId>
                <groupId>org.slf4j</groupId>
            </exclusion>
        </exclusions>
    </dependency>

    <!--Kafka Streams API Client Dependency-->
    <dependency>
        <groupId>org.apache.kafka</groupId>
        <artifactId>kafka-streams</artifactId>
        <version>1.1.0</version>
        <!--Excluded in Map-->
        <exclusions>
            <exclusion>
                <artifactId>slf4j-api</artifactId>
                <groupId>org.slf4j</groupId>
            </exclusion>
        </exclusions>
    </dependency>

    <!--Avro Serialization Dependency-->
    <dependency>
        <groupId>org.apache.avro</groupId>
        <artifactId>avro</artifactId>
        <version>1.8.2</version>
        <!--Excluded in Map-->
        <exclusions>
            <exclusion>
                <artifactId>slf4j-api</artifactId>
                <groupId>org.slf4j</groupId>
            </exclusion>
        </exclusions>
    </dependency>

    <!--Avro pojo class generation Dependency-->
    <dependency>
        <groupId>org.apache.avro</groupId>
        <artifactId>avro-maven-plugin</artifactId>
        <version>1.8.2</version>
        <!--Excluded in Map-->
        <exclusions>
            <exclusion>
                <artifactId>slf4j-api</artifactId>
                <groupId>org.slf4j</groupId>
            </exclusion>
        </exclusions>
    </dependency>

    <!--Avro Serialization Dependency-->
    <dependency>
        <groupId>org.apache.avro</groupId>
        <artifactId>avro-tools</artifactId>
        <version>1.8.2</version>
        <!--Excluded in Map-->
        <exclusions>
            <exclusion>
                <artifactId>slf4j-api</artifactId>
                <groupId>org.slf4j</groupId>
            </exclusion>
            <exclusion>
                <artifactId>commons-logging</artifactId>
                <groupId>commons-logging</groupId>
            </exclusion>
        </exclusions>
    </dependency>

</dependencies>

And I have excluded anything related to slf4j or even logging at this point from my dependency map which currently looks like this: Deps-Map

My Application logging is working as expected (Console and RolingFileAppender) but I just can't seem to eliminate this stubborn little Warning.

Thanks everyone for reading my little book here; especially if you have any suggestions!

DVS
  • 783
  • 7
  • 25
  • While not ideal solutions I had a couple ideas: 1) Use the statement `org.apache.log4j.BasicConfigurator.configure();` to provide default log4j config as detailed here: https://stackoverflow.com/questions/1140358/how-to-initialize-log4j-properly But this has the side effect of dumping dependency log messages to the console output. 2) Provide a log4j.xml config with a console appender and set the root log level to `OFF` to disable the output. This works without the console clutter of the first option, but now I have a extraneous file in my repo. Still hoping for a better solution... – DVS Jul 19 '18 at 11:40

1 Answers1

0

Ultimately I went with option 2 from my comment above.

The reason for this was that in the end it seemed more important to keep log messages minimal and clear in production over an additional file in source control. Additionally the 2nd option of a separate log4j configuration file allowed me to easily turn on additional debugging details from the dependencies pretty easily if I ever needed them such as my KStreams app just doesn't seem to work as expected and I get stumped.

Specifically, I actually created two versions of the log4j configuration so that I had different configurations for local and (tokenized) release versions like my other configuration/properties like so:

$ ls src/main/resources/*/*.xml
src/main/resources/local/log4j.xml*   src/main/resources/release/log4j.xml*
src/main/resources/local/log4j2.xml*  src/main/resources/release/log4j2.xml*

Then I setup log4j.xml as normal but defaulted the root logger level to off so it silences the logs unless changed and looked something like so:

<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">
<log4j:configuration debug="false">

    <appender name="MyConsoleAppender" class="org.apache.log4j.ConsoleAppender">
        <layout class="org.apache.log4j.PatternLayout">
            <param name="ConversionPattern"
                   value="%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %m%n" />
        </layout>
    </appender>

    <appender name="MyRollingFileAppender" class="org.apache.log4j.RollingFileAppender">
        <param name="append" value="true" />
        <param name="maxFileSize" value="100MB" />
        <param name="maxBackupIndex" value="10" />
        <param name="file" value="c:\\Logs\\KStreamsApp\\Dependencies\\KStreamsAppDependencies.log" />
        <layout class="org.apache.log4j.PatternLayout">
            <param name="ConversionPattern"
                   value="%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %m%n" />
        </layout>
    </appender>

    <root>
        <level value="Off" />
        <appender-ref ref="MyConsoleAppender" />
        <appender-ref ref="MyRollingFileAppender" />
    </root>

</log4j:configuration>
DVS
  • 783
  • 7
  • 25