You're going to need a Java compiler for that. The Eclipse Compiler for Java was written in Java, and can run as a standalone compiler, so it should be pretty easy to either hook into another entry point or add another entry point. I'm not sure how hard it would be to make it compile against the current classpath (classloader, really) rather than rt.jar.
There's also broadening your requirements a bit and using a Groovy interpreter (Groovy's at least a superset of Java), or even switch to Javascript (Java ships with a JS engine) or Ruby.