3

I have detected a severe memory leak in a piece of code that is used for compiling and running Java code in run time. I have created heap dumps and it seems com.sun.tools.javac.util.SharedNameTable$NameImpL is the culprit.

What I'd like to know is how I can prevent SharedNameTable from taking up so much space. Is there a way to force free the SharedNameTable ?

enter image description here

Compiler code:

public static Object compile(String code) throws IOException, InvocationTargetException, IllegalAccessException, ClassNotFoundException, NoSuchMethodException, java.lang.InstantiationException {
    String uuid = UUID.randomUUID().toString();
    File theDir = new File(uuid);

    if (!theDir.exists()) {
        System.out.println("creating directory: " + uuid);
        boolean result = false;

        try{
            theDir.mkdir();
            result = true;
        }
        catch(SecurityException se){
            System.out.println(se);
        }
        if(result) {
            System.out.println("DIR created");
        }
    }
    //Compile
    JavaCompiler jc = ToolProvider.getSystemJavaCompiler();
    StandardJavaFileManager sjfm = jc.getStandardFileManager(null, null, null);
    JavaFileObject javaObjectFromString = getJavaFileContentsAsString(new StringBuilder(code));
    System.out.println(javaObjectFromString.getName());
    Iterable fileObjects = Arrays.asList(javaObjectFromString);
    String[] options = new String[]{"-d", "/src/"+uuid};
    DiagnosticCollector<JavaFileObject> diagnostics = new DiagnosticCollector<JavaFileObject>();
    if (jc.getTask(null, null, diagnostics, Arrays.asList(options), null, fileObjects).call()) {
        sjfm.close();
        System.out.println("Class has been successfully compiled");
        //Load
        URL[] urls = new URL[]{new URL("file:///src/"+uuid+"/")};
        URLClassLoader ucl = new URLClassLoader(urls);
        Class cl = ucl.loadClass("TestClass");
        System.out.println("Class has been successfully loaded");
        //Run
        final Method method = cl.getDeclaredMethod("testMethod");
        final Object object = cl.newInstance();
        ExecutorService executor = Executors.newFixedThreadPool(1);
        Callable<Object> task = new Callable<Object>() {
            public Object call() throws IllegalAccessException, InvocationTargetException {
                return method.invoke(object);
            }
        };
        Future<Object> future = executor.submit(task);
        try {
            Object result = future.get(20, TimeUnit.SECONDS);
            return result;
        } catch (TimeoutException ex) {
            return "Method timed out (20 seconds). Please review your code.";
        } catch (InterruptedException e) {
            return "Method interrupted.";
        } catch (ExecutionException e) {
            e.printStackTrace();
            return String.format("ExecutionException thrown! %s", e.getMessage());
        } finally {
            future.cancel(true);
            executor.shutdown();
        }
    }
    sjfm.close();
    System.out.println("Class compilation failed!");
    //Create diagnostics and return them
    List<String> errors = new ArrayList<>();
    for (Diagnostic diagnostic : diagnostics.getDiagnostics()){
        String s = String.format("[Code:%s] %s on line %d / column % d in %s",
                diagnostic.getCode(),
                diagnostic.getKind().toString(),
                diagnostic.getLineNumber(),
                diagnostic.getColumnNumber(),
                diagnostic.getSource());
        errors.add(s);
        errors.add(diagnostic.toString());
    }
    return errors.toArray();
}

Edit:

I have found a similar question where SoftReferences are pointed out to be the cause of the problem. Forcing an OutOfMemoryException should force clear these. However I'm running this code within an node.js app using the 'node-java' npm package. When I try to force an OoM, my node.js app will be killed before the Exception is thrown.

Community
  • 1
  • 1
Hartger
  • 329
  • 1
  • 14
  • What happen if you leave the if statment empty just calling `jc.getTask(null, null, diagnostics, Arrays.asList(options), null, fileObjects).call()` ?? – Fran Montero Nov 05 '15 at 15:25
  • I'm sorry, I'm not sure if I understood your suggestion correctly. I tried running `jc.getTask(null, null, diagnostics, Arrays.asList(options), null, fileObjects).call()` outside the if statement and it had no result on memory consumption. – Hartger Nov 05 '15 at 15:43
  • So compilation task isn't the problem. – Fran Montero Nov 05 '15 at 15:50
  • Is this really a memory leak? Soft references will be collected if there is a need for freeing up memory. – biziclop Nov 06 '15 at 09:25
  • Node.JS seems to fail before memory is freed up. I have found a [reported bug](https://bugs.openjdk.java.net/browse/JDK-8039262) in Java with the exact same memory leak as the one I am experiencing (`SharedNameTable` and `ZipFileIndex` increasing in size when using `javax.tools.JavaCompiler$CompilationTask.call`). – Hartger Nov 06 '15 at 10:30
  • You can control how quickly soft referenced objects are cleaned out via the `-XX:SoftRefLRUPolicyMSPerMB=` flag, where the default value of `` is 1000, and lower values mean faster eviction. (The amount of free heap in megabytes is multiplied by this number to give you the time in milliseconds after which unused soft references are collected.) – biziclop Nov 06 '15 at 11:11
  • Possible duplicate of [Dynamic Compiling Without Create Physical File](http://stackoverflow.com/questions/10118380/dynamic-compiling-without-create-physical-file) – Paul Sweatte Jan 20 '17 at 19:06

1 Answers1

0

As I wrote in my answer to this question, there is a compiler option which prevents the use of SharedNameTable altogether. You can just add it as an additional compiler option:

String[] options = new String[]{"-d", "/src/"+uuid, "-XDuseUnsharedTable"};
rolve
  • 10,083
  • 4
  • 55
  • 75