Java ClassLoading from client's memory

Question

I am developing a client-server application using Java Sockets where at some point the client has to download some classes from the server (through the already bound Socket) and use them. I am able to read the class file bytes and send them to the client. So then, the client has to use a ClassLoader to load the classes from memory. So, in my opinion, the problem is not really related to Sockets, but it is about custom class loading (you will see why I say that, later in this post).

My setup is as follows: the client project with a single package named client with a single class in it, named LiveClientTest and then the server project with a single package named server with 3 classes in it: ClientMain, LiveServerTest (the entry point) and MyStrings. In short, all server-side code is under the package server and all client-side under the client. Each of the two packages is in a different project also.

The problem occurs when the custom ClassLoader of the client (named LiveClientTest.MemoryClassLoader) tries to load a non-static nested class (named MyStrings.NestedNonStaticSubclass) which refers to its enclosing class (named MyStrings) before constructing the object (of type MyStrings.NestedNonStaticSubclass). Although the classes compile fine, the error appears at runtime, while loading the class MyStrings.NestedNonStaticSubclass with the custom ClassLoader. I know it sounds weird, but you can see what I mean if you take a look at the code.

Server side code:

LiveServerTest class (entry point):

package server;

import java.io.DataOutputStream;
import java.io.File;
import java.io.IOException;
import java.io.OutputStream;
import java.net.ServerSocket;
import java.net.Socket;
import java.net.URISyntaxException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.Objects;
import javax.swing.JOptionPane;

public class LiveServerTest {
    
    //Convert class binary name representation to file path:
    public static String classNameToResourcePath(final String className) {
        return className.replace('.', File.separatorChar) + ".class";
    }
    
    //Get directory location of the given class (packages not included):
    public static Path getDirectoyPath(final Class clazz) throws URISyntaxException {
        return new File(clazz.getProtectionDomain().getCodeSource().getLocation().toURI()).toPath();
    }
    
    //Get absolute file location of the given class:
    public static Path getClassFilePath(final Class c) throws URISyntaxException {
        return getDirectoyPath(c).resolve(classNameToResourcePath(c.getName()));
    }
    
    public static void main(final String[] args) {
        ServerSocket srv = null;
        try {
            srv = new ServerSocket(Integer.parseInt(Objects.requireNonNull(JOptionPane.showInputDialog(null, "Enter host port number:"))));
            System.out.println("Waiting for client connection...");
            try (final Socket sck = srv.accept();
                 final OutputStream os = sck.getOutputStream()) {
                srv.close();
                srv = null;
                
                //These are the classes we need the client to load:
                final Class[] clientClasses = new Class[] {
                    ClientMain.class,
                    MyStrings.class,
                    MyStrings.NestedStatic.class,
                    MyStrings.NestedNonStaticSubclass.class
                };
                
                System.out.println("Sending all client classes' bytes to client...");
                final DataOutputStream dos = new DataOutputStream(os);
                dos.writeInt(clientClasses.length);
                for (final Class clazz: clientClasses) {
                    final byte[] contents = Files.readAllBytes(getClassFilePath(clazz));
                    dos.writeUTF(clazz.getName());
                    dos.writeInt(contents.length);
                    dos.write(contents);
                }
                
                System.out.println("Main application now starts...");
                //Here would go the server side code for the client-server application.
            }
        }
        catch (final IOException | URISyntaxException | RuntimeException x) {
            JOptionPane.showMessageDialog(null, x.toString());
        }
        finally {
            System.out.println("Done.");
            try { if (srv != null) srv.close(); } catch (final IOException iox) {}
        }
    }
}

ClientMain class:

package server;

import java.io.DataInputStream;
import java.net.Socket;

public class ClientMain {
    //This method is called by the client to start the application:
    public static void main(final Socket sck,
                            final DataInputStream dis,
                            final ClassLoader loader,
                            final String[] args) {
        System.out.println("Running...");
        //Here would go the client side code for the client-server application.
        
        //Just test that all the classes are loaded successfully:
        System.out.println(new MyStrings("A", "B", "C").new NestedNonStaticSubclass().getFunction().apply(2)); //Should print "C".
    }
}

MyStrings class:

package server;

import java.util.function.IntFunction;

public class MyStrings {
    
    public static class NestedStatic {
        private final IntFunction<String> f;
        
        public NestedStatic(final IntFunction<String> f) {
            this.f = f;
        }
        
        public IntFunction<String> getFunction() {
            return f;
        }
    }
    
    //This class produces the error when loaded:
    public class NestedNonStaticSubclass extends NestedStatic {
        public NestedNonStaticSubclass() {
            super(i -> getString(i)); //Here we refer to MyStrings object before constructing the NestedNonStaticSubclass object.
        }
    }
    
    private final String[] array;
    
    public MyStrings(final String... array) {
        this.array = array.clone();
    }
    
    public String getString(final int i) {
        return array[i];
    }
}

Client side code:

package client;

import java.io.DataInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.lang.reflect.InvocationTargetException;
import java.net.Socket;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.Iterator;
import java.util.Objects;
import javax.swing.JOptionPane;

public class LiveClientTest {
    
    public static class MemoryClassLoader extends ClassLoader {
        
        //The key is the name of the class, and value is the compiled class file bytes:
        private final HashMap<String, byte[]> classByteCode = new HashMap<>();

        @Override
        public /*synchronized*/ Class findClass(final String name) throws ClassNotFoundException {
            try {
                return super.findClass(name);
            }
            catch (final ClassNotFoundException cnfx) {
                if (!classByteCode.containsKey(name))
                    throw new ClassNotFoundException(name);
                final byte[] byteCode = classByteCode.get(name);
                return defineClass(name, byteCode, 0, byteCode.length);
            }
        }

        //Try to load all classes that are downloaded (with readClass):
        ArrayList<String> loadClasses() {
            final ArrayList<String> classNames = new ArrayList<>(classByteCode.keySet());
            int oldSize;
            do {
                oldSize = classNames.size();
                final Iterator<String> classNamesIter = classNames.iterator();
                while (classNamesIter.hasNext()) {
                    final String className = classNamesIter.next();
                    try {
                        loadClass(className);
                        classNamesIter.remove();
                    }
                    catch (final ClassNotFoundException x) {
                    }
                }
            }
            while (classNames.size() < oldSize); //If we reached a point where there are some classes that can not be loaded, then quit...
            return classNames; //Return the classes that failed to be loaded (if any) (should be empty).
        }
        
        //Read class bytes from supplied DataInputStream:
        void readClass(final DataInputStream dis) throws IOException {
            final String name = dis.readUTF();
            final byte[] contents = new byte[dis.readInt()];
            int i = 0, n = dis.read(contents);

            //Make sure all 'contents.length' multitude of bytes are read:
            while (n >= 0 && (i + n) < contents.length) {
                i += n;
                n = dis.read(contents, i, contents.length - i);
            }
            if (/*n < 0 ||*/ (i + n) != contents.length)
                throw new IOException("Unfinished class input (" + name + " - " + contents.length + ").");

            classByteCode.put(name, contents);
        }
    }
    
    public static void main(final String[] args) {
        try {
            final String host = Objects.requireNonNull(JOptionPane.showInputDialog(null, "Enter host name or address:"));
            final int port = Integer.parseInt(Objects.requireNonNull(JOptionPane.showInputDialog(null, "Enter host port number:")));
            try (final Socket sck = new Socket(host, port);
                 final InputStream is = sck.getInputStream()) {
                final DataInputStream dis = new DataInputStream(is);
                final MemoryClassLoader loader = new MemoryClassLoader();
                
                //Download all classes and put them into the class loader:
                System.out.println("Downloading...");
                for (int i = dis.readInt(); i > 0; --i)
                    loader.readClass(dis);
                
                //Load all downloaded classes from the class loader:
                System.out.println("Loading...");
                System.out.println("Failed to load: " + loader.loadClasses() + '.');

                //Call main method in main downloaded class:
                loader
                .loadClass("server.ClientMain") //package is from server side.
                .getDeclaredMethod("main", Socket.class, DataInputStream.class, ClassLoader.class, String[].class)
                .invoke(null, sck, dis, loader, args);
            }
        }
        catch (final IOException | ClassNotFoundException | NoSuchMethodException | IllegalAccessException | InvocationTargetException | RuntimeException x) {
            x.printStackTrace();
            JOptionPane.showMessageDialog(null, x);
        }
    }
}

Client side output:

Downloading...
Loading...
Exception in thread "main" java.lang.ClassFormatError: Illegal field name "server.MyStrings$this" in class server/MyStrings$NestedNonStaticSubclass
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:642)
    at client.LiveClientTest$MemoryClassLoader.findClass(LiveClientTest.java:30)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at client.LiveClientTest$MemoryClassLoader.loadClasses(LiveClientTest.java:44)
    at client.LiveClientTest.main(LiveClientTest.java:89)

So my question is:

Why does the code fail with a ClassFormatError, what does that mean, and how to avoid it in this particular scenario?

My question is not:

What alternatives exist? (such as using a URLClassLoader, or alternative ways of class loading from memory other than a custom ClassLoader, etc...)

How to reproduce:

1. I am using JRE 1.8.0_251 (and I would like this to work for 1.8), so I think you must put the source files in different projects (one for the client and one for the server) in order to make sure that the client does not already have direct visibility of the server's classes while class-loading them.
2. Run the server's main class server.LiveServerTest and input a port number for the server in the dialog that pops up. Then, run the client's main class client.LiveClientTest and enter localhost for the host (first dialog that pops up) and then the port number of the server (second dialog that pops up).
3. The stack trace will be in your CLI (through System.err) and not in a GUI.
4. The code will not work if you build the projects into jar files, but it is written to work for plain class files into directories, for simplicity. For example, in NetBeans, don't build into jar files, but rather click Run File for each entry point.
5. The built classes should have their file's .class extention in lower case.

Warning:

If you put the source files in different packages but in the same project, or even worse in the same package, then the class loading might succeed without errors because:

1. I am using the default constructor of the ClassLoader class in my LiveClientTest.MemoryClassLoader class, which means the parent class loader is the system class loader.
2. The LiveClientTest.MemoryClassLoader.findClass method first searches the parent ClassLoader and then, if that fails, it searches the downloaded classes. To my knowledge, this is the suggested way of implementing this, mainly because the ClassLoader class (which is the parent class of my LiveClientTest.MemoryClassLoader class) caches already defined classes.

References:

1. How to load JAR files dynamically at Runtime?
2. In which scenarios the remote class loading are needed?
3. Does the Java ClassLoader load inner classes?
4. Java - Get a list of all Classes loaded in the JVM
5. Java: How to load a class (and its inner classes) that is already on the class path?
6. Create new ClassLoader to reload Class
7. How to use classloader to load class file from server to client
8. Custom Java classloader and inner classes
9. ClassFormatError in java 8?
10. JVM Invalid Nested Class Name?
11. https://bugs.openjdk.java.net/browse/JDK-8145051
12. https://www.programming-books.io/essential/java/implementing-a-custom-classloader-0f0fe95cf7224c668e631a671eef3b94
13. https://www.baeldung.com/java-classloaders
14. https://www.infoworld.com/article/2077260/learn-java-the-basics-of-java-class-loaders.html
15. https://www.oracle.com/technical-resources/articles/javase/classloaders.html
16. https://www.ibm.com/developerworks/library/j-onejar/index.html

I am new to class loading, so please don't take my words for granted.

Huge post, because of the divided code of the MRE. Sorry. I tried to make the code as minimal as possible.