Class.forName(name, instantiation, classLoader) doesn't add class to classpath

Question

I'm generating .java class files at the runtime and need to utilize those classes inside the code instantly. So I compile the .java classes using Compiler API to make .class files:

JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();

DiagnosticCollector<JavaFileObject> diagnostics = new DiagnosticCollector<>();

StandardJavaFileManager manager = compiler.getStandardFileManager(diagnostics, null, null);

File file = new File("path to file");

Iterable<? extends JavaFileObject> sources = manager.getJavaFileObjectsFromFiles(Arrays.asList(file));

CompilationTask task = compiler.getTask(null, manager, diagnostics, null, null, sources);

task.call();

manager.close();

Then I need to get references to those compiled classes using Class.forName(), But if I just call Class.forName("com.foo.Bar") it throws ClassNotFoundException, Assuming it's because the new .class files aren't added to classpath I looked for the methods of adding classes to the classpath at runtime. I encountered some ambiguities related to this concept:

1. Is this approach (of compiling the .java file first, using compiler API, and add it to class loader at the second step) correct? To be able utilizing the class in the code instantly.

2. AFAIK, There are 2 methods to dynamically load classes into classpath at runtime: one is using a custom ClassLoader like this: (which I had error to compile as it complained that BuiltinClassLoader doesn't have addURL method):

    // Get the ClassLoader class
    ClassLoader cl = ClassLoader.getSystemClassLoader();
    Class<?> clazz = cl.getClass();

    // Get the protected addURL method from the parent URLClassLoader class
    Method method = clazz.getSuperclass().getDeclaredMethod("addURL", new Class[] { URL.class });

    // Run projected addURL method to add JAR to classpath
    method.setAccessible(true);
    method.invoke(cl, new Object[] { cls });

Another method is using Class.forName(name, instantiation, classLoader) to add a class to the classpath (which gives the class reference at the same too). The first method I wasn't able to apply since I got a compiler error (Java 11) as mentioned above. Regarding the second method, Would Class.forName(name, instantiation, classLoader) attach the new classes to the classpath if we call the default class loader like this? :

Class.forName("com.foo.Bar",true, ClassLoader.getSystemClassLoader());
// or:
Class.forName("com.foo.Bar",true, ApiHandler.class.getClassLoader());

It doesn't work for me. Which variation of the above classLoader arguments are correct and why do these don't work? Is it mandotary to create a custom classloader and pass it to the Class.forName()?

3. I'm making the .java files inside the com.foo package in src folder of the eclipse project. Their compiled .class files are also generated at the same folder (using compiler API). When I refresh project using eclipse (right-click on the project -> Refresh) the related .class files would be generated in the target/classes folder and that's when the classes could be accessed through the code (e.g using Class.forName("com.foo.Bar). May it be that if I produce .class files (by compiler API) in the target/classes folder, The classes would be recognizable without the need of introducing them to the classpath?

---

UPDATE:

I was able to use the compiled classes in my code, By saving the respected .class files in the target/classes folder, mentioned in the 3rd question above) of the project. (By adding -d option to the compiler's getTask() method:

Iterable<String> options = Arrays.asList( new String[] { "-d", System.getProperty("user.dir") + "/target/classes/"} );
.
.
.

CompilationTask task = compiler.getTask(null, manager, diagnostics, options, null, sources);

This way, It seems the classes are not even required to be added to the classpath using classLoader; as the class is accessible using a simple Class.forName(). How Do you explain this?

Class<?> cls1 = Class.forName("com.foo.Bar");

And also with through the ClassLoader way, of course:

ClassLoader classLoader = ClassLoader.getSystemClassLoader(); 

Class<?> cls = classLoader.loadClass("com.foo.Bar");

Answer 1

The safest solution is to create a new ClassLoader implementation and load the generated classes via the new loader, like shown in this answer.

But since Java 9, there is the possibility to define classes within your own context, i.e. within the same package, if no class with that name has been defined/loaded yet. Such a class definition may even supersede an definition on the class path, as said, as long as it has not been loaded yet. So not only subsequent Class.forName(String) calls will get resolved to this class definition but even non-reflective references.

This can be demonstrated with the following program.

class Dummy { // to make the compiler happy
    static native void extensionMethod();
}
public class CompileExtension {
    public static void main(String[] args) throws IOException, IllegalAccessException {
        // customize these, if you want, null triggers default behavior
        DiagnosticListener<JavaFileObject> diagnosticListener = null;
        Locale locale = null;

        // the actual class implementation, to be present at runtime only
        String class1 =
            "class Dummy {\n"
          + "    static void extensionMethod() {\n"
          + "        System.out.println(\"hello from dynamically compiled code\");\n"
          + "    }\n"
          + "}";
        JavaCompiler c = ToolProvider.getSystemJavaCompiler();
        StandardJavaFileManager fm
          = c.getStandardFileManager(diagnosticListener, locale, Charset.defaultCharset());
        // define where to store compiled class files - use a temporary directory
        fm.setLocation(StandardLocation.CLASS_OUTPUT,
            Set.of(Files.createTempDirectory("compile-test").toFile()));
        JavaCompiler.CompilationTask task = c.getTask(null, fm,
            diagnosticListener, Set.of(), Set.of(),
            Set.of(new SimpleJavaFileObject(
                URI.create("string:///Class1.java"), JavaFileObject.Kind.SOURCE) {
                    public CharSequence getCharContent(boolean ignoreEncodingErrors) {
                        return class1;
                    }
                }));

        if(task.call()) {
            FileObject fo = fm.getJavaFileForInput(
                StandardLocation.CLASS_OUTPUT, "Dummy", JavaFileObject.Kind.CLASS);
            // these are the class bytes of the first class
            byte[] classBytes = Files.readAllBytes(Paths.get(fo.toUri()));
            MethodHandles.lookup().defineClass(classBytes);

            Dummy.extensionMethod();
        }
    }
}

The Dummy definition only exists to be able to insert an invocation to the desired method at compile-time, whereas at runtime, the dynamically defined class takes its place, before the method gets invoked.

But handle with care. As said, the custom class loader is the safest solution. Normally, you should create compile-time references to extensions via an interface which is always present and only load implementations dynamically, which can be cast to the interface at runtime and then used through the API defined by the interface.