Convert String to Code

Question

I want to know if there is any way to convert a String to Java compilable code.

I have a comparative expression saved in a database field. I want to retrieve it from database, then evaluate it inside a conditional structure.

Is there any way to do this?

Answer 1

If you are using Java 6, you could try the Java Compiler API. At its core is the JavaCompiler class. You should be able to construct the source code for your Comparator object in memory.

Warning: I have not actually tried the code below as the JavaCompiler object is not available on my platform, for some odd reason...

Warning: Compiling arbitrary Java code can be hazardous to your health.

Consider yourself warned...

String comparableClassName = ...; // the class name of the objects you wish to compare
String comparatorClassName = ...; // something random to avoid class name conflicts
String source = "public class " + comparatorClassName + " implements Comparable<" + comparableClassName + "> {" +
                "    public int compare(" + comparableClassName + " a, " + comparableClassName + " b) {" +
                "        return " + expression + ";" +
                "    }" +
                "}";

JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();

/*
 * Please refer to the JavaCompiler JavaDoc page for examples of the following objects (most of which can remain null)
 */
Writer out = null;
JavaFileManager fileManager = null;
DiagnosticListener<? super JavaFileObject> diagnosticListener = null;
Iterable<String> options = null;
Iterable<String> classes = null;
Iterable<? extends JavaFileObject> compilationUnits = new ArrayList<? extends JavaFileObject>();
compilationUnits.add(
    new SimpleJavaFileObject() {
        // See the JavaDoc page for more details on loading the source String
    }
);

compiler.getTask(out, fileManager, diagnosticListener, options, classes, compilationUnits).call();

Comparator comparator = (Comparator) Class.forName(comparableClassName).newInstance();

After this, you just have to store the appropriate Java expression in your database field, referencing a and b.

Answer 2

_{The question of how to programmatically compile Java Code that is given as a String is asked quite frequently and in various forms, sometimes referring to code that is stored in a database or entered by the user. When I searched for information about this, I stumbled upon many of these questions, and was disappointed to see that the general recommendation was to use external tools (BeanShell, Groovy...). The answer by Adam Paynter to this question was the most helpful in order to at least figure out the relevant keywords. But even by consulting further external resources (like an example from Java2s), I struggled with implementing a pure in-memory compilation of one or more Java classes (that actually worked) using only the JavaCompiler API.}

---

So here is an example showing the whole process of compiling one or multiple classes in-memory, at runtime, when their source code is given as a string. It is built around a small utility class, RuntimeCompiler, that simply receives a sequence class names and the corresponding source codes, and then allows compiling these classes and obtaining the Class objects.

It is a MCVE that can be compiled and executed directly - with a JDK, not with a JRE, because the latter does not contain the tools like the JavaCompiler.

import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.OutputStream;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.lang.reflect.Modifier;
import java.net.URI;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.Comparator;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;

import javax.tools.Diagnostic;
import javax.tools.DiagnosticCollector;
import javax.tools.FileObject;
import javax.tools.ForwardingJavaFileManager;
import javax.tools.JavaCompiler;
import javax.tools.JavaCompiler.CompilationTask;
import javax.tools.JavaFileObject;
import javax.tools.JavaFileObject.Kind;
import javax.tools.SimpleJavaFileObject;
import javax.tools.StandardJavaFileManager;
import javax.tools.ToolProvider;

/**
 * An example showing how to use the RuntimeCompiler utility class
 */
public class RuntimeCompilerExample
{
    public static void main(String[] args) throws Exception
    {
        simpleExample();
        twoClassExample();
        useLoadedClassExample();
    }

    /**
     * Simple example: Shows how to add and compile a class, and then
     * invoke a static method on the loaded class.
     */
    private static void simpleExample()
    {
        String classNameA = "ExampleClass";
        String codeA =
            "public class ExampleClass {" + "\n" + 
            "    public static void exampleMethod(String name) {" + "\n" + 
            "        System.out.println(\"Hello, \"+name);" + "\n" + 
            "    }" + "\n" + 
            "}" + "\n";

        RuntimeCompiler r = new RuntimeCompiler();
        r.addClass(classNameA, codeA);
        r.compile();

        MethodInvocationUtils.invokeStaticMethod(
            r.getCompiledClass(classNameA), 
            "exampleMethod", "exampleParameter");
    }

    /**
     * An example showing how to add two classes (where one refers to the 
     * other), compile them, and invoke a static method on one of them
     */
    private static void twoClassExample()
    {
        String classNameA = "ExampleClassA";
        String codeA =
            "public class ExampleClassA {" + "\n" + 
            "    public static void exampleMethodA(String name) {" + "\n" + 
            "        System.out.println(\"Hello, \"+name);" + "\n" + 
            "    }" + "\n" + 
            "}" + "\n";

        String classNameB = "ExampleClassB";
        String codeB =
            "public class ExampleClassB {" + "\n" + 
            "    public static void exampleMethodB(String name) {" + "\n" + 
            "        System.out.println(\"Passing to other class\");" + "\n" + 
            "        ExampleClassA.exampleMethodA(name);" + "\n" + 
            "    }" + "\n" + 
            "}" + "\n";

        RuntimeCompiler r = new RuntimeCompiler();
        r.addClass(classNameA, codeA);
        r.addClass(classNameB, codeB);
        r.compile();

        MethodInvocationUtils.invokeStaticMethod(
            r.getCompiledClass(classNameB), 
            "exampleMethodB", "exampleParameter");
    }

    /**
     * An example that compiles and loads a class, and then uses an 
     * instance of this class
     */
    private static void useLoadedClassExample() throws Exception
    {
        String classNameA = "ExampleComparator";
        String codeA =
            "import java.util.Comparator;" + "\n" + 
            "public class ExampleComparator " + "\n" + 
            "    implements Comparator<Integer> {" + "\n" + 
            "    @Override" + "\n" + 
            "    public int compare(Integer i0, Integer i1) {" + "\n" + 
            "        System.out.println(i0+\" and \"+i1);" + "\n" + 
            "        return Integer.compare(i0, i1);" + "\n" + 
            "    }" + "\n" + 
            "}" + "\n";

        RuntimeCompiler r = new RuntimeCompiler();
        r.addClass(classNameA, codeA);
        r.compile();

        Class<?> c = r.getCompiledClass("ExampleComparator");
        Comparator<Integer> comparator = (Comparator<Integer>) c.newInstance();

        System.out.println("Sorting...");
        List<Integer> list = new ArrayList<Integer>(Arrays.asList(3,1,2));
        Collections.sort(list, comparator);
        System.out.println("Result: "+list);
    }

}


/**
 * Utility class for compiling classes whose source code is given as
 * strings, in-memory, at runtime, using the JavaCompiler tools.
 */
class RuntimeCompiler
{
    /**
     * The Java Compiler
     */
    private final JavaCompiler javaCompiler;

    /**
     * The mapping from fully qualified class names to the class data
     */
    private final Map<String, byte[]> classData;

    /**
     * A class loader that will look up classes in the {@link #classData}
     */
    private final MapClassLoader mapClassLoader;

    /**
     * The JavaFileManager that will handle the compiled classes, and
     * eventually put them into the {@link #classData}
     */
    private final ClassDataFileManager classDataFileManager;

    /**
     * The compilation units for the next compilation task
     */
    private final List<JavaFileObject> compilationUnits;


    /**
     * Creates a new RuntimeCompiler
     * 
     * @throws NullPointerException If no JavaCompiler could be obtained.
     * This is the case when the application was not started with a JDK,
     * but only with a JRE. (More specifically: When the JDK tools are 
     * not in the classpath).
     */
    public RuntimeCompiler()
    {
        this.javaCompiler = ToolProvider.getSystemJavaCompiler();
        if (javaCompiler == null)
        {
            throw new NullPointerException(
                "No JavaCompiler found. Make sure to run this with "
                    + "a JDK, and not only with a JRE");
        }
        this.classData = new LinkedHashMap<String, byte[]>();
        this.mapClassLoader = new MapClassLoader();
        this.classDataFileManager =
            new ClassDataFileManager(
                javaCompiler.getStandardFileManager(null, null, null));
        this.compilationUnits = new ArrayList<JavaFileObject>();
    }

    /**
     * Add a class with the given name and source code to be compiled
     * with the next call to {@link #compile()}
     * 
     * @param className The class name
     * @param code The code of the class
     */
    public void addClass(String className, String code)
    {
        String javaFileName = className + ".java";
        JavaFileObject javaFileObject =
            new MemoryJavaSourceFileObject(javaFileName, code);
        compilationUnits.add(javaFileObject);
    }

    /**
     * Compile all classes that have been added by calling 
     * {@link #addClass(String, String)}
     * 
     * @return Whether the compilation succeeded
     */
    boolean compile()
    {
        DiagnosticCollector<JavaFileObject> diagnosticsCollector =
            new DiagnosticCollector<JavaFileObject>();
        CompilationTask task =
            javaCompiler.getTask(null, classDataFileManager,
                diagnosticsCollector, null, null, 
                compilationUnits);
        boolean success = task.call();
        compilationUnits.clear();
        for (Diagnostic<?> diagnostic : diagnosticsCollector.getDiagnostics())
        {
            System.out.println(
                diagnostic.getKind() + " : " + 
                diagnostic.getMessage(null));
            System.out.println(
                "Line " + diagnostic.getLineNumber() + 
                " of " + diagnostic.getSource());
            System.out.println();
        }
        return success;
    }


    /**
     * Obtain a class that was previously compiled by adding it with
     * {@link #addClass(String, String)} and calling {@link #compile()}. 
     * 
     * @param className The class name
     * @return The class. Returns <code>null</code> if the compilation failed.
     */
    public Class<?> getCompiledClass(String className)
    {
        return mapClassLoader.findClass(className);
    }

    /**
     * In-memory representation of a source JavaFileObject 
     */
    private static final class MemoryJavaSourceFileObject extends
        SimpleJavaFileObject
    {
        /**
         * The source code of the class
         */
        private final String code;

        /**
         * Creates a new in-memory representation of a Java file
         * 
         * @param fileName The file name
         * @param code The source code of the file
         */
        private MemoryJavaSourceFileObject(String fileName, String code)
        {
            super(URI.create("string:///" + fileName), Kind.SOURCE);
            this.code = code;
        }

        @Override
        public CharSequence getCharContent(boolean ignoreEncodingErrors)
            throws IOException
        {
            return code;
        }
    }

    /**
     * A class loader that will look up classes in the {@link #classData}
     */
    private class MapClassLoader extends ClassLoader
    {
        @Override
        public Class<?> findClass(String name)
        {
            byte[] b = classData.get(name);
            return defineClass(name, b, 0, b.length);
        }
    }

    /**
     * In-memory representation of a class JavaFileObject
     * @author User
     *
     */
    private class MemoryJavaClassFileObject extends SimpleJavaFileObject
    {
        /**
         * The name of the class represented by the file object
         */
        private final String className;

        /**
         * Create a new java file object that represents the specified class
         * 
         * @param className THe name of the class
         */
        private MemoryJavaClassFileObject(String className)
        {
            super(URI.create("string:///" + className + ".class"), 
                Kind.CLASS);
            this.className = className;
        }

        @Override
        public OutputStream openOutputStream() throws IOException
        {
            return new ClassDataOutputStream(className);
        }
    }


    /**
     * A JavaFileManager that manages the compiled classes by passing
     * them to the {@link #classData} map via a ClassDataOutputStream
     */
    private class ClassDataFileManager extends
        ForwardingJavaFileManager<StandardJavaFileManager>
    {
        /**
         * Create a new file manager that delegates to the given file manager
         * 
         * @param standardJavaFileManager The delegate file manager
         */
        private ClassDataFileManager(
            StandardJavaFileManager standardJavaFileManager)
        {
            super(standardJavaFileManager);
        }

        @Override
        public JavaFileObject getJavaFileForOutput(final Location location,
            final String className, Kind kind, FileObject sibling)
            throws IOException
        {
            return new MemoryJavaClassFileObject(className);
        }
    }


    /**
     * An output stream that is used by the ClassDataFileManager
     * to store the compiled classes in the  {@link #classData} map
     */
    private class ClassDataOutputStream extends OutputStream
    {
        /**
         * The name of the class that the received class data represents
         */
        private final String className;

        /**
         * The output stream that will receive the class data
         */
        private final ByteArrayOutputStream baos;

        /**
         * Creates a new output stream that will store the class
         * data for the class with the given name
         * 
         * @param className The class name
         */
        private ClassDataOutputStream(String className)
        {
            this.className = className;
            this.baos = new ByteArrayOutputStream();
        }

        @Override
        public void write(int b) throws IOException
        {
            baos.write(b);
        }

        @Override
        public void close() throws IOException
        {
            classData.put(className, baos.toByteArray());
            super.close();
        }
    }
}

/**
 * Utility methods not directly related to the RuntimeCompiler
 */
class MethodInvocationUtils
{
    /**
     * Utility method to invoke the first static method in the given 
     * class that can accept the given parameters.
     *  
     * @param c The class
     * @param methodName The method name
     * @param args The arguments for the method call
     * @return The return value of the method call
     * @throws RuntimeException If either the class or a matching method
     * could not be found
     */
    public static Object invokeStaticMethod(
        Class<?> c, String methodName, Object... args)
    {
        Method m = findFirstMatchingStaticMethod(c, methodName, args);
        if (m == null)
        {
            throw new RuntimeException("No matching method found");
        }
        try
        {
            return m.invoke(null, args);
        }
        catch (IllegalAccessException e)
        {
            throw new RuntimeException(e);
        }
        catch (IllegalArgumentException e)
        {
            throw new RuntimeException(e);
        }
        catch (InvocationTargetException e)
        {
            throw new RuntimeException(e);
        }
        catch (SecurityException e)
        {
            throw new RuntimeException(e);
        }
    }

    /**
     * Utility method to find the first static method in the given
     * class that has the given name and can accept the given 
     * arguments. Returns <code>null</code> if no such method 
     * can be found.
     * 
     * @param c The class
     * @param methodName The name of the method 
     * @param args The arguments
     * @return The first matching static method.
     */
    private static Method findFirstMatchingStaticMethod(
        Class<?> c, String methodName, Object ... args)
    {
        Method methods[] = c.getDeclaredMethods();
        for (Method m : methods)
        {
            if (m.getName().equals(methodName) &&
                Modifier.isStatic(m.getModifiers()))
            {
                Class<?>[] parameterTypes = m.getParameterTypes();
                if (areAssignable(parameterTypes, args))
                {
                    return m;
                }
            }
        }
        return null;
    }

    /**
     * Returns whether the given arguments are assignable to the
     * respective types
     * 
     * @param types The types
     * @param args The arguments
     * @return Whether the arguments are assignable
     */
    private static boolean areAssignable(Class<?> types[], Object ...args)
    {
        if (types.length != args.length)
        {
            return false;
        }
        for (int i=0; i<types.length; i++)
        {
            Object arg = args[i];
            Class<?> type = types[i];
            if (arg != null && !type.isAssignableFrom(arg.getClass()))
            {
                return false;
            }
        }
        return true;
    }

}

EDIT In response to the comment:

In order to compile classes that are contained in external JAR files, it should be sufficient to add the JAR to the classpath of the calling application. The JavaCompiler will then pick up this classpath to find classes that it needs for the compilation.

_{There seems to be some magic involved. At least, I have not figured out the exact mechanisms behind that, just tested it with an example}

And a side note: Of course, one could consider literally arbitrary extensions to this class. My goal was to create a simple, standalone, easily copy-and-pastable example that shows the whole process, and may even be "useful" for certain application patterns.

For more sophisticated functionalities, one could consider either extending this class accordingly, or have a look, for example, at the Java-Runtime-Compiler from the OpenHFT project (I stumbled upon this a few weeks after I had written this answer). It basically uses the same techniques internally, but in a somewhat more sophisticated way, and also offers dedicated mechanisms for handling class loaders for external dependencies.

Answer 3

Use Groovy!

Binding binding = new Binding();
GroovyShell shell = new GroovyShell(binding);
Object value = shell.evaluate("for (x=0; x<5; x++){println "Hello"}; return x");

Answer 4

You can use BeanShell.
The key class is bsh.Interpreter.

Here is a simple example:

Interpreter interpreter = new Interpreter();
Object res = interpreter.eval("your expresion");

It is even possible to define a whole class instead of a single expression.

Answer 5

You can by using something like BeanShell.

Answer 6

You can't because java is a compiled language.

You, however, should use a javax.script api to execute code in runtime. JVM6 ships with Rhino (javascript interpreter) avaliable via javax.script.

http://java.sun.com/javase/6/docs/api/javax/script/package-summary.html

There are javax.script-compatible java interpreters (and bean shell) avaliable.

https://scripting.dev.java.net/

Answer 7

StringEscapeUtils.escapeJava from Commons Lang may help if you want to generate a piece of compilable code and dump a string there.

Answer 8

It's not fair to say that this is impossible. It is a very similar problem to the problem Java Server Pages (JSPs) have - in their case, there is code embedded in HTML files that needs to be compiled into a servlet and executed. If you really wanted to use that mechanism, I'm relatively sure you could dig through the source for a servlet container and figure out how they did it (probably even reusing their mechanism to some degree.)

However; it isn't an easy problem to solve (once you solve the obvious, immediate problem, you end up having to deal with issues with classloading and related problems.)

It certainly would seem to be a better idea to go with the Java Scripting Platform in JDK6.

Answer 9

Apparently Java Scripting Platform is better for this situation but You can also use Java Compiler Api. It provides methods for compiling java source files from within java code. In your case you can make a temporary file containing a class with your comparative expression then you can load that file and use it. Off course this isn't very elegant. Check out http://www.juixe.com/techknow/index.php/2006/12/13/java-se-6-compiler-api/ for details on using Java Compiler Api

Answer 10

Groovy might also be an option for you.

It integrates cleanly with the Bean Scripting Framework, can be embedded directly quite easily and might be ok, syntax-wise, for you.

Answer 11

Yes it is possible in many ways.

As some have mentioned above, Java 6 allows you to parse, manipulate and rewrite code as it is loaded!

The solution can vary:

You could for example, write your DB expression as a Java class and insert your serialized class into the DB as a glob or blob or whatever its called.

Or, you could use a template to write a Java class to a file and insert your expression into it. Then compile the class at run-time (like a JSP to Servlet) and then dynamically load the class.

Of course you'd want to cache your compiled classes for future use if they have not been edited in the database.

Then there is also the option of using an integrated scripting engine such as most of the responses have mentioned.

Whatever you choose, maybe you'll be able to update this post with details of your choice, implementation, problems, notes, comments, etc.

Answer 12

You'd be much better off mapping your database condition info to an object and either implementing an equals method on it that you can use with BeanPropertyValueEqualsPredicate from the apache commons beanutils or implement Comparable instead. This should do what you're after without all the magic compiler nastiness or security issues of converting strings to code.

Answer 13

A simple way to get code snippets to executable byte code is with the Javassist library.

You can possibly adapt the techniques described in http://www.ibm.com/developerworks/java/library/j-dyn0610/ to fit your needs.

Answer 14

If you're getting the condition from a database, I'll wager there's a good chance that you might be wanting to use that condition to access data in that database.

If you're using an ORM such as JPA or Hibernate (traditional or JPA), you might be able to formulate a dynamic query expression that you'd pass to the createQuery() method. This isn't as good as being able to do an on-the-fly compile of arbitrary Java code, but maybe it's all you need, and that particular solution doesn't require any special inclusions or actions, since the query language compiler is part of the ORM system itself.

Of course, if you DO do dynamic queries that way, I'd recommend logging them somehow, since it can be a real pain to figure out what went wrong after the fact if your query string is now in the garbage collector.

Answer 15

You shouldn't. Really!

Are you inventing another enterprise rules engine?. You might want to read these links.

Consider the fact that the only people that is skilled enough to write code and then insert it into a database, probably are having an editor and a compiler anyway...

The compiler will catch all those pesky syntax errors and you can even test the code! Remember that editors and compilers, and even computer languages were invented to help the programmer to comfortably write comprehensible code with a reasonable effort.

While I'm at it: read about the complicators gloves too!

Answer 16

If all you really need to do is evaluate an expression stored in a database, you might want to look at JEP (Java Expression Parser)

The latest (commercial) version is here.

A slightly older, GPL version is here

Some examples for usage.

Answer 17

If you're willing to sacrifice the "Java code" portion of your requirement, you could use the Java Mathematic Expression Evaluator library. It allows you to specify a math expression (as a java.lang.String), add values for the variables, and then evaluate the expression.

I've used it in production code with great success.

Answer 18

I have used both BeanShell and GroovyShell but performance wise, GroovyShell is way faster if you prase and cache the script.