SLF4J MDC Memory Leak

Question

I've had a google for this, looked at multiple suggestions and nothing seems to help.

I have a JAX-RS application which using MDC, when an endpoint is hit sets a transactionId in order to make debugging easier. However, when i stop or restart Tomcat the logs are filled with entries like this:

27-Sep-2014 09:42:14.858 SEVERE [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoader.checkThreadLocalMapForLeaks The web application [/core-1.0.0-RC2] created a ThreadLocal with key of type [org.apache.log4j.helpers.ThreadLocalMap] (value [org.apache.log4j.helpers.ThreadLocalMap@464437fc]) and a value of type [java.util.Hashtable] (value [{siteCode=000tst, transactionId=dc8f3a1b-1d7a-4f91-abf6-58d015632d03}]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.

I have a RequestFilter where MDC is called:

import org.slf4j.MDC;

import javax.ws.rs.container.ContainerRequestContext;
import javax.ws.rs.container.ContainerRequestFilter;
import java.io.IOException;
import java.util.UUID;

public void filter(ContainerRequestContext containerRequestContext) throws IOException {

    String siteCodeHeader = containerRequestContext.getHeaderString("Site-Code");

    if (siteCodeHeader != null) {
        MDC.put("siteCode", siteCodeHeader);
    } else {
        MDC.put("siteCode", "NULL");
    }
    MDC.put("transactionId", UUID.randomUUID().toString());


}

These are my sl4fj dependencies:

<dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>slf4j-api</artifactId>
    <version>1.7.7</version>
</dependency>
<dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>slf4j-log4j12</artifactId>
    <version>1.7.7</version>
</dependency>

If I have a ResponseFilter with MDC.clear() It removes the values from the MDC, but doesn't seem to clear the thread:

27-Sep-2014 09:12:58.216 SEVERE [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoader.checkThreadLocalMapForLeaks The web application [/core-1.0.0-RC2] created a ThreadLocal with key of type [org.apache.log4j.helpers.ThreadLocalMap] (value [org.apache.log4j.helpers.ThreadLocalMap@391216c7]) and a value of type [java.util.Hashtable] (value [{}]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.

Apparently it was fixed in log4j 1.2.17 but the changes don't seem to have filtered through to slf4j.

score 2 · Accepted Answer · answered Sep 28 '14 at 15:04

Using the mix of the previous two answers I was able to fix the problem.

I used the log4j implementation of MDC rather than SLF4J, and added a ResponseFilter as well to do the clear out. It may, or may not have affected it, but I also used provider annotations rather than stipulating the classes in the web.xml.

RequestFilter (much the same):

package com.example.jaxrs;

import org.apache.log4j.MDC;

import javax.ws.rs.container.ContainerRequestContext;
import javax.ws.rs.container.ContainerRequestFilter;
import javax.ws.rs.ext.Provider;
import java.io.IOException;
import java.util.UUID;

@Provider
public class TransactionIdentifierRequestFilter implements ContainerRequestFilter {

    @Override
    public void filter(ContainerRequestContext containerRequestContext) throws IOException {

        String siteCodeHeader = containerRequestContext.getHeaderString("Site-Code");

        if (siteCodeHeader != null) {
            MDC.put("siteCode", siteCodeHeader);
        } else {
            MDC.put("siteCode", "NULL");
        }
        MDC.put("transactionId", UUID.randomUUID().toString());

    }
}

ResponseFilter:

package com.example.jaxrs;

import org.apache.log4j.MDC;

import javax.ws.rs.container.ContainerRequestContext;
import javax.ws.rs.container.ContainerResponseContext;
import javax.ws.rs.container.ContainerResponseFilter;
import javax.ws.rs.ext.Provider;
import java.io.IOException;

@Provider
public class TransactionIdentifierResponseFilter implements ContainerResponseFilter {

    @Override
    public void filter(ContainerRequestContext containerRequestContext, ContainerResponseContext containerResponseContext) throws IOException {
        MDC.clear();
    }
}

web.xml

<init-param>
    <param-name>jersey.config.server.provider.packages</param-name>
    <param-value>com.example.jaxrs</param-value>
</init-param>

This is the wrong way to go. You now have tied your code to Log4J. You should have cleared the MDC in the first place by [SLF4J means](http://www.slf4j.org/api/org/slf4j/MDC.html#clear%28%29). — Michael-O, Sep 28 '14 at 15:18
It didn't work. SLF4J MDC is built on an older version of log4j before the 1.2.17 fix was put. Discovered that by looking at the actual sources.The underlying logging framework i was using for slf4j was log4j, so didn't require any new dependencies etc or complete logging overhaul. — Crazy Dino, Sep 28 '14 at 15:42

score 1 · Answer 2 · answered Sep 27 '14 at 13:59

1

MDCs are hold in the current thread using ThreadLocal. If you add values in a filter, you have to remove them after the service call.

public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) throws IOException, ServletException {
   try
   {

     //add your mdcs

     // proceed along the chain
     chain.doFilter(request, response);

  } 
  finally
  {

     //remove your mdcs

  }
}

answered Sep 27 '14 at 13:59

Hannes

2,018
25
32

I've created a class that extends ServletContainer, overwritten the doFilter() method and set the servlet class in Spring to my custom class, however the doFilter doesn't appear to be hit. Do I need to explicitly call doFilter somehow? – Crazy Dino Sep 27 '14 at 17:49
Before servlet 3.0 spec, one would add a filter configuration to the web.xml file. Have a look at http://stackoverflow.com/questions/11092421/spring-dispatcherservlet-code-to-execute-before-it – Hannes Sep 27 '14 at 19:09

score 0 · Answer 3 · answered Sep 27 '14 at 13:52

0

This is clearly not a SLF4J problem but a Log4J problem only. Add the fixed Log4J version as a direct dependency to your POM with runtime scope and your problem should go away.

answered Sep 27 '14 at 13:52

Michael-O

18,123
6
55
121

Did not work. I'm using the SLF4j version of MDC not Log4J and the latest version of log4j is a dependency for another dependency so was already present. – Crazy Dino Sep 27 '14 at 14:46
then you missed to clean out the MDC after the request has been processed. – Michael-O Sep 27 '14 at 17:38
I have cleaned out the MDC after and returns that although the values have been cleared the threadlocal is still remaining. – Crazy Dino Sep 27 '14 at 18:38
Try logback and see whether you still have the memory leak. – Michael-O Sep 27 '14 at 18:47
adding log4j dependency version 1.2.17 solved the problem – Nov 15 '15 at 12:58

SLF4J MDC Memory Leak

3 Answers3