How to load JAR files dynamically at Runtime in JDK11+?

Question

I have an application that loads jar files in runtime dynamically using the following solution:

File file = ...
URL url = file.toURI().toURL();

URLClassLoader classLoader = (URLClassLoader)ClassLoader.getSystemClassLoader();
Method method = URLClassLoader.class.getDeclaredMethod("addURL", URL.class);
method.setAccessible(true);
method.invoke(classLoader, url);

This was done using the answer at How to load JAR files dynamically at Runtime?

I now want a solution that works JDK11+ that is equivalent to the original solution I used. Thus doing it programmatically without the need for third-party libraries/frameworks or loading/invoking single classes.

I tried the following:

1. Created a DynamicClassLoader that extends the UrlClassLoader:

public final class DynamicClassLoader extends URLClassLoader {


    public DynamicClassLoader(URL[] urls, ClassLoader parent) {
        super(urls, parent);
    }

    public DynamicClassLoader(String name, ClassLoader parent) {
        super(name, new URL[0], parent);
    }
    
    public DynamicClassLoader(ClassLoader parent) {
        this("classpath", parent);
    }

    public void addURL(URL url) {
        super.addURL(url);
    }
    
}

2. I then start my application wit java.system.class.loader flag:

java -Djava.system.class.loader=com.example.DynamicClassLoader

Then I have a jar file as a path object which I call with the following method:

public void loadJar(Path path) throws Exception {
        
        URL url = path.toUri().toURL();
        
        DynamicClassLoader classLoader = (DynamicClassLoader)ClassLoader.getSystemClassLoader();
        Method method = DynamicClassLoader.class.getDeclaredMethod("addURL", URL.class);
        method.setAccessible(true);
        method.invoke(classLoader, url);        
    }

When this method is called I get the following cast class exception:

class jdk.internal.loader.ClassLoaders$AppClassLoader cannot be cast to class com.example.classloader.DynamicClassLoader (jdk.internal.loader.ClassLoaders$AppClassLoader is
 in module java.base of loader 'bootstrap'; com.example.classloader.DynamicClassLoader is in unnamed module of loader 'app')

I am using Spring Boot (2.4) on OpenJDK11 (build 11.0.10+9).

Answer 1

Changing the ClassLoader of a running application is a terrible idea. Specially changing the system ClassLoader as you're trying to do.

The JVM loads classes lazily as it executes, then caches them on first load. Interfering with that process may cause serious issues for your application stability.

Using reflection to access internal methods of URLClassLoader, as you're doing, will just cause your application to become tied to particular JDK versions where your code happens to work, but it's not guaranteed to keep working as you upgrade to newer JDK versions, or even JDK from different vendors.

There are ways to do this though, but it's far from trivial.

For example, OSGi is a system that adds this feature to the JVM, but you must change your application to fit into the OSGi model, which is not very easy. One of its main attractions is exactly that it allows you to install bundles (jars with OSGi metadata), remove them, and even upgrade them all without restarting your application.

It does that by isolating all jars into their own ClassLoaders (to simplify things a bit) and calculating which classes are exported/imported by each bundle, then making sure that each ClassLoader can see the necessary classes from others as necessary (which is hard!).

You can have a simple dynamic loading mechanism, though, by creating new instances of URLClassLoader for each group of jars you want to load at runtime.

It works like this, approximately:

URL[] jars = { ... };

// you can pass a parent loader to the constructor...
// it will delegate first, so already loaded classes are not tried to reload
var loader = new URLClassLoader(jars);

// once you have a ClassLoader, you may use it to load classes provided
// by the new jars
Class<?> type = loader.loadClass("DynamicClass");

// do something with this class?
// But you must cast it to some type that's known in the current class-path,
// otherwise this class is only usable by reflection!

// For example, if you know this class must implement Runnable and it has
// a default constructor, you may be able to do this:
var runnable = (Runnable) type.newInstance();

// as we now know the type, we can actually use it type-safely
runnable.run();

Answer 2

Based on the discussion in the comment section, I was able to find a solution. This maybe not as generic as I hoped for and assumes that you know the class to use (opposite to just add the Maven dependency to your pom.xml or the old JDK8 solution).

1. Download the Maven dependencies (using Jeka)

List<Path> paths = resolveDependency(groupId, artifactId, version);

public List<Path> resolveDependency(String groupId, String artifactId, String version) throws Exception {
        
        String dependency = groupId + ":" + artifactId + ":" + version;
        
        JkDependencySet deps = JkDependencySet.of()
                .and(dependency)
                .withDefaultScopes(COMPILE_AND_RUNTIME);

        JkDependencyResolver resolver = JkDependencyResolver.of(JkRepo.ofMavenCentral());
        List<Path> paths = resolver.resolve(deps, RUNTIME).getFiles().getEntries();

        return paths;

    }

2. Load the jar files

List<Class> classes = loadDependency(paths);

public List<Class> loadDependency(List<Path> paths) throws Exception {

        List<Class> classes = new ArrayList<>();

        for(Path path: paths){

            URL url = path.toUri().toURL();
            URLClassLoader child = new URLClassLoader(new URL[] {url}, this.getClass().getClassLoader());

            ArrayList<String> classNames = getClassNamesFromJar(path.toString());

            for (String className : classNames) {
                Class classToLoad = Class.forName(className, true, child);
                classes.add(classToLoad);
            }
        }

        return classes;

    }


    // Returns an arraylist of class names in a JarInputStream
    private ArrayList<String> getClassNamesFromJar(JarInputStream jarFile) throws Exception {
        ArrayList<String> classNames = new ArrayList<>();
        try {
            //JarInputStream jarFile = new JarInputStream(jarFileStream);
            JarEntry jar;

            //Iterate through the contents of the jar file
            while (true) {
                jar = jarFile.getNextJarEntry();
                if (jar == null) {
                    break;
                }
                //Pick file that has the extension of .class
                if ((jar.getName().endsWith(".class"))) {
                    String className = jar.getName().replaceAll("/", "\\.");
                    String myClass = className.substring(0, className.lastIndexOf('.'));
                    classNames.add(myClass);
                }
            }
        } catch (Exception e) {
            throw new Exception("Error while getting class names from jar", e);
        }
        return classNames;
    }


// Returns an arraylist of class names in a JarInputStream
// Calls the above function by converting the jar path to a stream
private ArrayList<String> getClassNamesFromJar(String jarPath) throws Exception {
        return getClassNamesFromJar(new JarInputStream(new FileInputStream(jarPath)));
    }

3. Use the class

Like Renato points out, know you need to know the class to use it. In my case it's a Camel component which I need to cast and add to this framework. The classes are what you retrieved in the second step and the scheme is the name of the component.

Component camelComponent = getComponent(classes, scheme);
context.addComponent(scheme, camelComponent);

public Component getComponent(List<Class> classes, String scheme) throws Exception {

        Component component = null;
        for(Class classToLoad: classes){
            String className = classToLoad.getName().toLowerCase();
            if(className.endsWith(scheme + "component")){
                Object object =  classToLoad.newInstance();
                component = (Component) object;
            }
        }

        return component;
    }

Thus, the second part is to make it dynamically available. Added the first and third part to have an example of a complete solution.