0

I am trying to upgrade the log4j version in kafka and zookeeper docker image from 1.x.x to 2.x.x

Docker commands

RUN rm /zookeeper-3.5.9/lib/log4j-1.2.17.jar

RUN wget https://repo1.maven.org/maven2/org/apache/logging/log4j/log4j-core/2.17.0/log4j-core-2.17.0.jar -P /zookeeper-3.5.9/lib/
RUN wget https://repo1.maven.org/maven2/org/apache/logging/log4j/log4j-api/2.17.0/log4j-api-2.17.0.jar -P /zookeeper-3.5.9/lib/

RUN rm /opt/kafka_2.12-2.2.1/libs/log4j-1.2.17.jar


RUN wget https://repo1.maven.org/maven2/org/apache/logging/log4j/log4j-core/2.17.0/log4j-core-2.17.0.jar -P /opt/kafka_2.12-2.2.1/libs/
RUN wget https://repo1.maven.org/maven2/org/apache/logging/log4j/log4j-api/2.17.0/log4j-api-2.17.0.jar -P /opt/kafka_2.12-2.2.1/libs/

This is the error I am getting while initialising the pod

Failed to instantiate SLF4J LoggerFactory Reported exception: java.lang.NoClassDefFoundError: org/apache/log4j/Level

Can someone please guide me on how can I upgrade the log4j version without upgrading the base kafka and zookeeper version?

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
priya dhana
  • 11

2 Answers2

0

I assume you are trying to manually patch log4j security issues on your own?

Don't.

Kafka and Zookeeper projects have already done this by moving to reload4j, which is still log4j 1.x compatible. (logback is used in Zookeeper 3.8, actually)

Upgrade to the latest version of each Zookeeper and Kafka (using existing Docker images, even), rather than cherry-pick JAR files that will not work with these projects without further source code changes.

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
  • Thanks for the reply. Yes we are trying to patch the log4j vulnerabilities manually. Unfortunately we have a dependency with these kafka and zookeeper version and cannot upgrade immediately, but we are planning to upgrade. Is there any way to upgrade the log4j jar to 2.x version without getting the error Failed to instantiate SLF4J LoggerFactory Reported exception: java.lang.NoClassDefFoundError: org/apache/log4j/Level. Please help. – priya dhana Apr 17 '23 at 03:35
  • No. There isn't. Kafka clients after version 0.9 are completely backwards compatible. Upgrade your brokers. Otherwise, try using reload4j jars instead, which actually are source-compatible. Or, find online ways to manually patch the log4j1.2 jar file – OneCricketeer Apr 17 '23 at 12:27
  • I understand. Can you please elaborate more on the reload4j jars options? Just replacing log4j jar with reload4j jar would work? Is there any particular version of reload4j to be replaced with log4j 1.2.17? Can reload4j be used with both kafka and zookeeper? – priya dhana Apr 17 '23 at 14:13
  • Latest version. Refer main page https://reload4j.qos.ch/ ; you can also add `slf4j-reload4j` and replace the slf4j-log4j12 JAR if that exists. Or, as mentioned, use the latest Kafka and Zookeeper images, which have already done this – OneCricketeer Apr 17 '23 at 18:35
  • 1
    To be precise Log4j 1.x has several withstanding security vulnerabilities (having been EOL since 2015), but was never affected by _Log4Shell_. – Piotr P. Karwasz Apr 17 '23 at 19:54
0

As far as I can tell, neither Kafka nor Zookeeper use Log4j 1.x nor Reload4j directly, but they access it through SLF4J (1.7.x).

You can easily switch SLF4J backends by:

  • in Kafka remove the SLF4J-to-Log4j 1.x bridge (slf4j-log4j12) and Log4j 1.x (log4j),
  • in Zookeeper remove the SLF4J-to-Reload4J bridge (slf4j-reload4j) and Reload4j (reload4j),
  • replace the above mentioned libraries with:
    1. SLF4J 1.7.x to Log4j 2.x API bridge: log4j-slf4j-impl
    2. Log4j 2.x API: log4j-api,
    3. Log4j 2.x Core: log4j-core.

You'll also need to put a configuration file on the classpath, since the configuration syntax changed between 1.x and 2.x.

Piotr P. Karwasz
  • 12,857
  • 3
  • 20
  • 43
  • 1
    `kafka-run-class` (the main entrypoint to start the server) does use `-Dlog4j.configuration` at start-up. https://github.com/apache/kafka/blob/trunk/bin/kafka-run-class.sh#L233 – OneCricketeer Apr 17 '23 at 20:44
  • @OneCricketeer: yes, that might need changing to `-Dlog4j2.configurationFile` or one might rely on the [automatic configuration](https://logging.apache.org/log4j/2.x/manual/configuration.html#AutomaticConfiguration). – Piotr P. Karwasz Apr 18 '23 at 05:56
  • You can use KAFKA_LOG4J_OPTS env var to set, yes. But that'll still require writing a new log file to disk, however, not the classpath. So simply replacing some jar files isn't the only step as the existing log4j.properties won't be read – OneCricketeer Apr 18 '23 at 12:37