Looking for a safe Sandbox API for JVM plugins

Question

When you have an application server, and you want to run third-party plugins in it, you can use a restrictive Security Manager to stop them doing things like System.exit(), but that's only half the story. Those untrusted plugins can still get in an infinite loop, or eat up all the free heap before you have time to blink. Thread.stop() has been deprecated, so you can't just kill an amok Thread, and since the heap is shared, not only will the plugin get an OutOfMemoryError when it uses up all the heap, but so will all other threads running.

Is there some Open Source application/API/framework out there that can manipulate the bytecode of plugin classes to make the threads killable and/or track the allocation so that the thread can be killed if it allocates too much? Even if the code is not readily "packaged" to be "used separately". You can make a Thread killable by inserting code that can produce an Exception at will, triggered by another "manager" thread, and make sure that Exception is not caught by the plugin. And you can add some kind of counters that count the number of calls and loops and amount of allocation, and have the "manager" thread kill a plugin that breaks the configured limits.

I think all those things could be done with ASM, but I'm hoping they have been done before. I could let the plugins run in their own JVM, but that would involve massive constant marshaling/unmarshaling of the data, and in case the plugin JVM dies/crashes, I still don't know which of the potentiall dozens (100s?) of plugins was the problem, and I can't possibly run one JVM per plugin.

I have found some related questions, but none solve the issue with infinite loops and eating the heap:

• Sandbox JVM to secure server from untrusted sources
• How do i create a java sandbox?
• SecurityManager for a cloud service “sandbox”

Answer 1

I found a very simple solution for the 'System.exec('rm -rf *')' problem:

https://svn.code.sf.net/p/loggifier/code/trunk/de.unkrig.commons.lang/src/de/unkrig/commons/lang/security/Sandbox.java

package de.unkrig.commons.lang.security;

import java.security.AccessControlContext;
import java.security.Permission;
import java.security.Permissions;
import java.security.ProtectionDomain;
import java.util.Collections;
import java.util.HashMap;
import java.util.Map;
import java.util.WeakHashMap;

import de.unkrig.commons.nullanalysis.Nullable;

/**
 * This class establishes a security manager that confines the permissions for code executed through specific classes,
 * which may be specified by class, class name and/or class loader.
 * <p>
 * To 'execute through a class' means that the execution stack includes the class. E.g., if a method of class {@code A}
 * invokes a method of class {@code B}, which then invokes a method of class {@code C}, and all three classes were
 * previously {@link #confine(Class, Permissions) confined}, then for all actions that are executed by class {@code C}
 * the <i>intersection</i> of the three {@link Permissions} apply.
 * <p>
 * Once the permissions for a class, class name or class loader are confined, they cannot be changed; this prevents any
 * attempts (e.g. of the confined class itself) to release the confinement.
 * <p>
 * Code example:
 * <pre>
 *  Runnable unprivileged = new Runnable() {
 *      public void run() {
 *          System.getProperty("user.dir");
 *      }
 *  };
 *
 *  // Run without confinement.
 *  unprivileged.run(); // Works fine.
 *
 *  // Set the most strict permissions.
 *  Sandbox.confine(unprivileged.getClass(), new Permissions());
 *  unprivileged.run(); // Throws a SecurityException.
 *
 *  // Attempt to change the permissions.
 *  {
 *      Permissions permissions = new Permissions();
 *      permissions.add(new AllPermission());
 *      Sandbox.confine(unprivileged.getClass(), permissions); // Throws a SecurityException.
 *  }
 *  unprivileged.run();
 * </pre>
 */
public final
class Sandbox {

    private Sandbox() {}

    private static final Map<Class<?>, AccessControlContext>
    CHECKED_CLASSES = Collections.synchronizedMap(new WeakHashMap<Class<?>, AccessControlContext>());

    private static final Map<String, AccessControlContext>
    CHECKED_CLASS_NAMES = Collections.synchronizedMap(new HashMap<String, AccessControlContext>());

    private static final Map<ClassLoader, AccessControlContext>
    CHECKED_CLASS_LOADERS = Collections.synchronizedMap(new WeakHashMap<ClassLoader, AccessControlContext>());

    static {

        // Install our custom security manager.
        if (System.getSecurityManager() != null) {
            throw new ExceptionInInitializerError("There's already a security manager set");
        }
        System.setSecurityManager(new SecurityManager() {

            @Override public void
            checkPermission(@Nullable Permission perm) {
                assert perm != null;

                for (Class<?> clasS : this.getClassContext()) {

                    // Check if an ACC was set for the class.
                    {
                        AccessControlContext acc = Sandbox.CHECKED_CLASSES.get(clasS);
                        if (acc != null) acc.checkPermission(perm);
                    }

                    // Check if an ACC was set for the class name.
                    {
                        AccessControlContext acc = Sandbox.CHECKED_CLASS_NAMES.get(clasS.getName());
                        if (acc != null) acc.checkPermission(perm);
                    }

                    // Check if an ACC was set for the class loader.
                    {
                        AccessControlContext acc = Sandbox.CHECKED_CLASS_LOADERS.get(clasS.getClassLoader());
                        if (acc != null) acc.checkPermission(perm);
                    }
                }
            }
        });
    }

    // --------------------------

    /**
     * All future actions that are executed through the given {@code clasS} will be checked against the given {@code
     * accessControlContext}.
     *
     * @throws SecurityException Permissions are already confined for the {@code clasS}
     */
    public static void
    confine(Class<?> clasS, AccessControlContext accessControlContext) {

        if (Sandbox.CHECKED_CLASSES.containsKey(clasS)) {
            throw new SecurityException("Attempt to change the access control context for '" + clasS + "'");
        }

        Sandbox.CHECKED_CLASSES.put(clasS, accessControlContext);
    }

    /**
     * All future actions that are executed through the given {@code clasS} will be checked against the given {@code
     * protectionDomain}.
     *
     * @throws SecurityException Permissions are already confined for the {@code clasS}
     */
    public static void
    confine(Class<?> clasS, ProtectionDomain protectionDomain) {
        Sandbox.confine(
            clasS,
            new AccessControlContext(new ProtectionDomain[] { protectionDomain })
        );
    }

    /**
     * All future actions that are executed through the given {@code clasS} will be checked against the given {@code
     * permissions}.
     *
     * @throws SecurityException Permissions are already confined for the {@code clasS}
     */
    public static void
    confine(Class<?> clasS, Permissions permissions) {
        Sandbox.confine(clasS, new ProtectionDomain(null, permissions));
    }

    // Code for 'CHECKED_CLASS_NAMES' and 'CHECKED_CLASS_LOADERS' omitted here.

}

Answer 2

I have been involved in adding support to plugins into my existing web application using OSGi framework. Based on my experience with working and reading up about this topic, this is what I have understood:

1) OSGi is the most famous and highly supported Plugin Standards on the JVM. There are multiple different implementation of this specification like Equinox (eclipse), Felix (Apache), Dynamic Modules (Spring) etc. So, a lot of big open source foundation work behind this.

2) There is nothing in the specification that talks about resource constraining. In fact, they have actively avoided talking about it. Its not like they do not know about it, but their stance is, on the JVM, there is nothing you can do to prevent people from doing some sort of harm. So, the gold standard of plugin spec on JVM does not talk about this.

There are bits and pieces of information (like the links you have posted) about how to implement some of these constraints, but there is nothing that you can do in terms of preventing malicious plugins from doing bad things.

That means, there is no catch-all way to stop resource hogging (CPU, Memory, File Descriptors, SQL connections etc).

Heap and CPU are the easy ones. What about just doing a "System.exec('rm -rf')"? Or open lets say 64000 sockets and potentially stop being able to create any new sockets.

There are just so many ways things can go wrong that trying to come up with an in-process sandbox for the JVM to allow plugins is almost impossible.