Java JAR protection

Question

I've coded a Java app and I plan to distribute it online. Each release will be locked with a secret serial key I made.

I need to secure my jar file from decompiler etc. Here is what I've done so far:

1. User enters his serial key into a form
2. The serial is sent to my dev server through a php script
3. The script generates a new jar bin file which is encrypted in AES 128
4. My "loader" downloads the jar file as bytes and decrypts it.
5. It invokes the main method.
6. User can use the app as he like to
7. User close the app
8. The cache is cleared and everything returns to step 1 or before.

I've made the steps 1 to 3, but I need to know if it is possible to make a custom classloader that grabs bytes from HTTP, decrypts them and invokes the main method. As the file is fully crypted (saved as bin on the PHP server), I can't use a basic class loader. About step 8, is it possible to unload content from the computer's memory?

Answer 1

Yes, it is possible, but it's also useless.

Your classloader will be the weakest point of your security system. It cannot be encrypted, thus it will be relatively easy to decompile it.

Now, as there is a place in the Classloader where you must have a decrypted class in a form of bytearray, the only thing an attacker will have to do, is to dump this byte array to a file.

Referring to dystroy's code:

classBytes = EncryptionUtil.decryptBytes(url, key); 

// In this moment, classBytes contains unencrypted class.
// If an attacker adds:   
//
//   FileOutputStream fos = new FileOutputStream("some.name");
//   fos.write(classBytes);
//   fos.close();
// 
// he will destroy all your security.


return defineClass(name, classBytes, 0, classBytes.length);

Answer 2

Yes, you can provide to a classloader the bytes you grab. I do it in a similar problem :

public class SecureClassLoader extends URLClassLoader {

    @Override
    protected Class<?> findClass(final String name) throws ClassNotFoundException {
        if (isEncrypted(name)) {
            final String resourceName = name.replace('.', '/') + ".class";
            URL url = findResource(resourceName);
            if (url != null) {
                byte[] classBytes = null;
                try {
                    classBytes = EncryptionUtil.decryptBytes(url, key);
                    return defineClass(name, classBytes, 0, classBytes.length);
                } catch (ClassFormatError e) {
                    log.severe("Bad format for decrypted class " + name);
                    throw new EncryptedClassNotFoundException(name, e);
                } catch (InvalidKeyException e) {
                    throw new EncryptedClassNotFoundException(name, e);
                } catch (IOException e) {
                    throw new EncryptedClassNotFoundException(name, e);
                }
            }
        }

        // default loader
        try {
            return super.findClass(name);
        } catch (ClassNotFoundException e) {
            throw new EncryptedClassNotFoundException(name, e);
        }
    }

    private boolean isEncrypted(String className) {
        /// some things
    }
}

But this won't be sufficient to protect your code. At the very least, don't use the same bytes for two users (you seem to do it). And obfuscate your code (I use proguard). This will protect you against ordinary hackers, not the best ones.