This question is old and i am a bit late for the party (though this question is not answered yet), but it is possible. From what i understand, this sounds like a job for Just Another Framework. Full disclosure: I am the developer of the Framework i am describing. I had a similar problem and wrote this framework to reuse this system, which worked for this kind of issue.
TL;DR
Do not use JAF, if you want your own, fully functional, perfect language.
Use JAF, if you just want to create a simple scripting part, that decouples parts of your code or if you want to evaluate "user input" scripts at Runtime (like in the topic of this question).
What I don't know is, is Java suitable for making such a scripting language?
You can do it with nearly every programming language. You should consider to use Javascript or Lua or [...] and put those into a file instead of creating a custom scripting language. Java has a place for those languages. You should also consider to maybe read .java or .class files from a folder and take those into your "Game-Engine". If you however want to provide your own language and no one can change your mind about it (that was my issue), JAF can help you with that.
JAF is a framework, which allows you to define custom scripting languages within Java, that may be evaluated and executed at Runtime. You give the Parser some Functions, Rules as well as the raw script and the result is a fixed set of instructions (in the form of consumers), that can be executed as many times as you like.
This is by no means another language build on top of java! It is meant to be a foundation for your language. It is only meant for creating small parts though. An example would be some sort of logic that should be interchangeable without stopping the application. This logic should be executed via small scripts and provided via a Database or (like in your case) from files located in a folder.
You can use a Parser, to parse "raw-scripts" that are created by the users. This parser takes Functions and Rules in the form of a Strategy-/Command-Pattern, which basically define your language. After parsing, a "Script"-Object is returned, that can be executed as many times as you want. You can also set values of the Script and therefor connect this Script to your real and running java-code.
The real thing i loved about this however is, that you can provide custom script elements through Rules and Functions, which basically directly connect to java. Just an example would be the printline function. This calls System.out.println();. As i said, just an example, but you can imagine that you can call any existing code from your Functions and Rules.
A (very) simple example of how to use this would look like this:
String rawScript = ... //your users script here
Parser parser = Parser.create(); // used to parse the script
parser.add(IOModule.getPackage()); // add packages of Functions and Rules
Script script;
try {
script = parser.parse(rawScript); // Create the Script
} catch (ParsingFailedException e) {
// Handle an error while parsing the script
// Do not continue, there is no script!
}
script.setValue("yourKey", /* Whatever should be available in your script */); // Inject outside variables for your Rules/Functions
try {
script.run(); // Execute the script
} catch (ExecutionFailedException e) {
// Handle an error at the scripts execution
}
I do not want to write a book about how to use this framework. Take a look at the wiki, it is better explained there.
The advantage of this is, that you can simply define your own language, by providing simple commands that parse specific pieces of the raw script. They are just "applied"; All the magic is hidden within the framework. It's explained a bit within the frameworks-wiki, but i am bad at explaining. So you can create your own little scripting language by providing the building-blocks of your language without requiring deeper knowledge of theoretical computer science.
It has some issues though. It is currently not compatibly with the JSR 223. Also, it is (of course) slower than playn java.
