How to free all resources after reading a JRT?

Question

I am trying to read the list of modules available in a given Java 9+ installation, given its Java Home, using the method described in How to extract the file jre-9/lib/modules?.

The solution works, but it appears that the resources allocated to read the content of the Java Runtime Image are never freed, causing a memory leak, observable with VisualVM for instance:

How can I fix the memory leak in the following reproduction?

package leak;

import java.net.URI;
import java.net.URL;
import java.net.URLClassLoader;
import java.nio.file.FileSystem;
import java.nio.file.FileSystems;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.Collections;
import java.util.Map;
import java.util.stream.Stream;

public class JrtfsLeak {
  public static void main(String[] args) throws Exception {
    Path javaHome = Paths.get(args[0]);
    for (int i = 0; i < 100000; ++i) {
      modules(javaHome).close();
    }
  }

  private static Stream<Path> modules(Path javaHome) throws Exception {
    Map<String, String> env = Collections.singletonMap("java.home", javaHome.toString());
    Path jrtfsJar = javaHome.resolve("lib").resolve("jrt-fs.jar");
    try (URLClassLoader classloader = new URLClassLoader(new URL[] { jrtfsJar.toUri().toURL() })) {
      try (FileSystem fs = FileSystems.newFileSystem(URI.create("jrt:/"), env, classloader)) {
        Path modulesRoot = fs.getPath("modules");
        return Files.list(modulesRoot);
      }
    }
  }
}

Answer 1

This is a JDK bug JDK-8260621 that has been fixed in JDK 17.
It was caused by a careless use of thread locals in ImageBufferCache.

Answer 2

Note that when you are running under Java 9 or newer, the underlying implementation will create a new class loader for the jrt-fs.jar when you specify the java.home option. So, these classes are not loaded by your URLClassLoader but a different class loader. When you don’t need support for versions prior to 9, you can omit the creation of a class loader.

In either case, they’re loaded by a custom class loader and could be unloaded when the garbage collector supports it. But the class jdk.internal.jimage.ImageBufferCache contains:

private static final ThreadLocal<BufferReference[]> CACHE =
    new ThreadLocal<BufferReference[]>() {
        @Override
        protected BufferReference[] initialValue() {
            // 1 extra slot to simplify logic of releaseBuffer()
            return new BufferReference[MAX_CACHED_BUFFERS + 1];
        }
    };

As explained in How does this ThreadLocal prevent the Classloader from getting GCed, a backreference from the value to the thread local can prevent its garbage collection and when the thread local is stored in a static variable, a reference to one of the classes loaded by the same class loader is enough.

And here, the value is an array of BufferReference, which means even when all entries of that array have been cleared, the array type itself has an implicit reference to the class loader of that filesystem.

But since its a thread local variable, we can work-around it by letting the key thread die. When I change your code to

public static void main(String[] args) throws InterruptedException {
    Path javaHome = Paths.get(args[0]);
    Runnable r = () -> test(javaHome);
    for(int i = 0; i < 1000; ++i) {
        Thread thread = new Thread(r);
        thread.start();
        thread.join();
    }
}

static void test(Path javaHome) {
    for (int i = 0; i < 1000; ++i) {
        try(var s = modules(javaHome)) {}
        catch(IOException ex) {
            throw new UncheckedIOException(ex);
        }
        catch(Exception ex) {
            throw new IllegalStateException(ex);
        }
    }
}

the classes get unloaded.

Answer 3

Refer to javadoc for method list (in class java.nio.file.Files). Here is the relevant part.

API Note:
This method must be used within a try-with-resources statement or similar control structure to ensure that the stream's open directory is closed promptly after the stream's operations have completed.

In other words you need to close the Stream returned by your modules method.