Anything you code in a Lua script can also be performed in C via the Lua C API. But it's harder to follow in C than the equivalent Lua script. When I have wanted to have some protected Lua scripts for a host program I have done things like for example:
- put them in a separate directory.
- Encrypt them into an archive that only my C program knows the private key for.
- Embedded them as multi-line strings in the C source code.
- Embed the compiled scripts as binary strings in my C source code so that they not as easily hack-able via the binary edit of program's exe. (Sometimes I use a SHA of the scripts to prevent them from being edited within the exe)
...and so on.
You definitely want to protect the computer running your game from malicious scripts by using sandboxes. E.g no file system access. But note, that to also protect your host program from all forms of malicious scripts (e.g. hanging the game's Lua instance thread via a while 1 loop; or huge memory use, complex string.match calls for e.g.) you have to do more complex work and add more constraints to the environment given to mod scripts.
E.g. even a while 1 do end will still hang the WOW UI.
Hope that helps.