Logging in AWS Lambda with slf4j

Question

I am using a lambda function and writing it in Java. I was looking up logging for Lambda functions when I read the docs and they support log4j - http://docs.aws.amazon.com/lambda/latest/dg/java-logging.html#java-wt-logging-using-log4j.

I was wondering if we could use logging using the Slf4j annotation as well since Slf4j is only a binding annotation. Has anybody tried using Slf4j before with lambda?

Answer 1

Yes, you can. Just add the following dependencies to your project:

    <dependency>
        <groupId>org.slf4j</groupId>
        <artifactId>jcl-over-slf4j</artifactId>
        <version>1.7.25</version>
    </dependency>
    <dependency>
        <groupId>org.slf4j</groupId>
        <artifactId>slf4j-log4j12</artifactId>
        <version>1.7.25</version>
    </dependency>
    <dependency>
        <groupId>com.amazonaws</groupId>
        <artifactId>aws-lambda-java-log4j</artifactId>
        <version>1.0.0</version>
    </dependency>

and create correct log4j.properties in /src/main/resources/ of your project, e.g.

log = .
log4j.rootLogger = DEBUG, LAMBDA

# Define the LAMBDA appender
log4j.appender.LAMBDA=com.amazonaws.services.lambda.runtime.log4j.LambdaAppender
log4j.appender.LAMBDA.layout=org.apache.log4j.PatternLayout
log4j.appender.LAMBDA.layout.conversionPattern=%d{yyyy-MM-dd HH:mm:ss} <%X{AWSRequestId}> %-5p %c{1}:%m%n

Answer 2

The jlib AWS Lambda Logback Appender allows you to use SLF4J with Logback from your AWS Lambda functions.

Simply add these dependencies:

Gradle (build.gradle)

dependencies {
    implementation 'org.slf4j:slf4j-api:1.8.0-beta2'
    runtimeOnly 'org.jlib:jlib-awslambda-logback:1.0.0'
}

Maven (pom.xml)

<dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>slf4j-api</artifactId>
    <version>1.8.0-beta2</version>
</dependency>
<dependency>
    <groupId>org.jlib</groupId>
    <artifactId>jlib-awslambda-logback</artifactId>
    <version>1.0.0</version>
    <scope>runtime</scope>
</dependency>

Then use the AwsLambdaAppender in your logging configuration:

Example XML configuration (src/main/resources/logback.xml)

<configuration>

    <appender name="awslambda" class="org.jlib.cloud.aws.lambda.logback.AwsLambdaAppender">
        <encoder type="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
            <pattern>[%d{yyyy-MM-dd HH:mm:ss.SSS}] &lt;%-36X{AWSRequestId:-request-id-not-set-by-lambda-runtime}&gt;
%-5level %logger{10} - %msg%n</pattern>
        </encoder>
    </appender>

    <root level="INFO">
        <appender-ref ref="awslambda" />
    </root>

</configuration>

Unlike other solutions, this Logback Appender correctly handles multi-line log messages, especially stack traces, and produces only a single CloudWatch Logs event per message.

The library also allows you to include the AWSRequestId, provided by the AWS Lambda runtime, in every single log message for better tracing.

While log4j2 requires additional handling in the build when you create an uber-jar, this solution works out of the box.

Disclaimer: I'm the developer of jlib

Answer 3

Just include following dependency:

<dependency>
    <groupId>io.symphonia</groupId>
    <artifactId>lambda-logging</artifactId>
    <version>1.0.0</version>
</dependency>

Source of lambda-logging code available at https://github.com/symphoniacloud/lambda-monitoring/tree/master/lambda-logging.

Background information available at: https://blog.symphonia.io/a-love-letter-to-lambda-logging-974b0eb49273

Answer 4

Ever since the AWS guide has been updated with details on Log4j 2 and SLF4J integration. The code snippets below are based on the guide with slf4j and log4j versions updated to the more recent ones.

pom.xml:

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-bom</artifactId>
            <version>2.19.0</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>
<dependencies>
    <dependency>
        <groupId>com.amazonaws</groupId>
        <artifactId>aws-lambda-java-log4j2</artifactId>
        <version>1.5.1</version>
        <scope>runtime</scope>
    </dependency>
    <dependency>
        <groupId>org.apache.logging.log4j</groupId>
        <artifactId>log4j-slf4j2-impl</artifactId>
    </dependency>
    <dependency>
        <groupId>org.slf4j</groupId>
        <artifactId>slf4j-api</artifactId>
        <version>2.0.6</version>
    </dependency>
</dependencies>

And src/main/resources/log4j2.xml:

<Configuration status="WARN">
    <Appenders>
        <Lambda name="Lambda">
            <PatternLayout>
                <pattern>%d{yyyy-MM-dd HH:mm:ss} %X{AWSRequestId} %-5p %c{1} - %m%n</pattern>
            </PatternLayout>
        </Lambda>
    </Appenders>
    <Loggers>
        <Root level="INFO">
            <AppenderRef ref="Lambda"/>
        </Root>
        <Logger name="software.amazon.awssdk" level="WARN"/>
        <Logger name="software.amazon.awssdk.request" level="DEBUG"/>
    </Loggers>
</Configuration>

Answer 5

Be aware that lambda requires the use of a shadowJar (shaded jar) which does not work correctly with log4j2 without an additional transformation, this is known albeit somewhat hard to grok issue...

To try to bring all these posts together and to clarify in order to enable faster resolution of this issue see below

• log4j JsonTemplateLayout doesn't work when jar file built by shadowJar
  • Original post that clued me in
• Console contains an invalid element or attribute "JsonTemplateLayout" even after adding dependency
  • Full example of how to fix in maven or gradle
• How correctly log AWS Lambda event with lambda powertools
  • Explanation with powertools
• Logging using Log4J2 on aws lambda - Class not found
  • Explanation where someone is claiming the issue is due to a bad path name (that is probably not the case)