How to create a custom ClassLoader for nested JAR files

Question

I am working with a Java library that has some nested JAR files in lib package.

I have 2 issues:

1. I cannot see referenced types in my IDE (I am using JetBrains IntelliJ)
2. Of course I get class not defined at runtime

I understand that I have to create and use a custom ClassLoader, will it solve both problems?
Is this the recommended way of achieving this result?

The JAR file is an Italian government provided library and I cannot modify it as it will be periodically updated as the regulation changes.

Answer 1

Yes, as far as I know, the standard ClassLoaders do not support nested JARs. Which is sad, since it would be a really nice idea, but Oracle just doesn't give a damn about it. Here is a 18-year old ticket:

https://bugs.java.com/bugdatabase/view_bug.do?bug_id=4735639

If you are getting those JARs from somebody else, the best thing would be to contact the vendor and ask them for a delivery in standards-compatible format. From your answer I realize that this might be difficult to achieve, but I would still try to talk to them, because it's the right thing to do. I'm pretty sure that everybody else in your position has the same issue. According to industry standards, such situation would usually hint your vendor into using Maven repository for their deliverables.

If talking to your vendor fails, you can re-pack the JARs as you get them. I would recommend writing an automated script for that and making sure it gets run on each delivery. You can either put all .class files into one uber-JAR, or just move the nested JARs outside the enclosing JAR. Caveat 1: there can be more than one class with the same name, so you need to make sure to take the correct one. Caveat 2: if the JARs were signed, you will lose the signature (unless you sign them with your own).

Option 3: you can always implement your own ClassLoader to load the classes from anywhere, even from the kitchen sink.

This guy did exactly this: https://www.ibm.com/developerworks/library/j-onejar/index.html

The short summary is that such a ClassLoader has to perform recursive unzipping, which is a bit of a pain-in-the-ass because archives are essentially made for stream access and not for random access, but apart from that it's perfectly doable.

You can use his solution as a "wrapper loader" which will replace your main class.

As far as IntelliJ IDEA goes, I don't believe it supports this functionality out-of-the box. The best thing would be either to re-package JARs as described above and add them as separate classpath entries, or to search if anybody has written a plugin for nested JAR support.

Answer 2

I don't know what you want to do after load jars. In my case, use jar dynamic loading for Servlet samples.

    try{
    final URLClassLoader loader = (URLClassLoader)ClassLoader.getSystemClassLoader();
    final Method method = URLClassLoader.class.getDeclaredMethod("addURL", URL.class);
    method.setAccessible(true);

    new File(dir).listFiles(new FileFilter() {
        @Override
        public boolean accept(File jar) {

            // load file if it is 'jar' type
            if( jar.toString().toLowerCase().contains(".jar") ){
                try {
                    method.invoke(loader, new Object[]{jar.toURI().toURL()});
                    XMLog.info_arr(logger, jar, " is loaded.");

                    JarInputStream jarFile = new JarInputStream(new FileInputStream(jar));
                    JarEntry jarEntry;

                    while (true) {
                        // load jar file
                        jarEntry = jarFile.getNextJarEntry();
                        if (jarEntry == null) {
                            break;
                        }

                        // load .class file in loaded jar file
                        if (jarEntry.getName().endsWith(".class")) {
                            Class loadedClass = Class.forName(jarEntry.getName().replaceAll("/", "\\.").replace(".class",""));

                            /*
                            * In my case, I load jar file for Servlet.
                            * If you want to use it for other case, then change below codes
                            */
                            WebServlet annotaions =  (WebServlet) loadedClass.getAnnotation(WebServlet.class);

                            // load annotation and mapping if it is Servlet
                            if (annotaions.urlPatterns().length > 0) {
                                ServletRegistration.Dynamic registration = servletContextEvent.getServletContext().addServlet(annotaions.urlPatterns().toString(), loadedClass);
                                registration.addMapping(annotaions.urlPatterns());
                            }
                        }
                    }
                } catch (Exception e) {
                    System.err.println("Can't load classes in jar");
                }
            }
            return false;
        }
    });
} catch(Exception e) {
    throw new RuntimeException(e);
}

Answer 3

Interestingly I just solved a version of this problem for JesterJ, though I had the additional requirement of loading dependencies for the code in the jar file as well. JesterJ (as of this evening's commits!) runs from a fat jar and accepts an argument denoting a second fat jar containing the classes, dependencies and configuration for a document ingestion plan (the user's code that I need to run).

The way my solution works is I borrow the knowledge of how to load jars inside of jars from Uno-Jar (the library that produces the fat jar), and stuff my own classloader in above it to control the evaluation order of the class loaders.

The key bit from https://github.com/nsoft/jesterj/blob/jdk11/code/ingest/src/main/java/org/jesterj/ingest/Main.java looks like this:

JesterJLoader jesterJLoader;

File jarfile = new File(javaConfig);
URL planConfigJarURL;
try {
  planConfigJarURL = jarfile.toURI().toURL();
} catch (MalformedURLException e) {
  throw new RuntimeException(e); // boom
}

jesterJLoader = (JesterJLoader) ClassLoader.getSystemClassLoader();

ClassLoader loader;
if (isUnoJar) {
  JarClassLoader jarClassLoader = new JarClassLoader(jesterJLoader, planConfigJarURL.toString());
  jarClassLoader.load(null);
  loader = jarClassLoader;
} else {
  loader = new URLClassLoader(new URL[]{planConfigJarURL}, jesterJLoader);
}
jesterJLoader.addExtLoader(loader);

My JesterJLoader is here:

https://github.com/nsoft/jesterj/blob/jdk11/code/ingest/src/main/java/org/jesterj/ingest/utils/JesterJLoader.java

Though if you are happy to simply delegate up and rely on existing classes on the main class path (rather than loading additional dependencies from the sub-fat-jar like I'm doing) yours could be much simpler. I go to a lot of effort to allow it to check the sub-jar first rather than delegating up to the parent immediately, and then have to keep track of what's been sent to the sub-jar to avoid loops and subsequent StackOverflowError...

Also note that the line where I get the system class loader is going to NOT be what you want, I'm also monkeying with the system loader to work around impolite things that some of my dependencies are doing with class loading.

If you decide to try to check out Uno-Jar pls note that resource loading for this nested scenario may yet be wonky and things definitely won't work before https://github.com/nsoft/uno-jar/commit/cf5af42c447c22edb9bbc6bd08293f0c23db86c2

Also: recently committed thinly tested code warning :)

Disclosure: I maintain both JesterJ and Uno-Jar (a fork of One-JAR the library featured in the link supplied by jurez) and welcome any bug reports or comments or even contributions!