Prevent Lua infinite loop

Question

I use lua interfaces to get lua support in my C# program, the worker thread will freeze if the user submits code like this

while true do end

I have a way to detect if a infinite loop is running, but I need a nice way of exiting the DoString method from the Worker thread. Any ideas?

edit: @kikito, Yes Im deteting it something like that. The problem I have is that I cant find a clean way of killing the DoString method, it looks like the Lua interfaces main class (Lua) has some static dependencies, because if I do lua.Close(); on my instance it will abort the DoString method, but the next time I instance an lua class new Lua(); it will crash saying something about protective memory

edit: A feature branch showing my .Close code https://github.com/AndersMalmgren/FreePIE/tree/detect-and-recover-infite-lua-loop

Answer 1

Sandboxing Lua

Setting hooks is not sufficient at all to prevent unintended waste of resources, let alone abuse- here's a simple example (the time is spent during string pattern matching- no debug hooks get called):

s=('a'):rep(20000):match('.-b')

The only reliable way to force time/memory constraints on a piece of Lua code is to run the Lua interpreter in a process of its own and to make your OS monitor that process.

The nice thing with Lua is that you won't need any complicated, OS-dependent permission setup for sandboxing: you just limit time and memory (reasonable; on windows there are Job Objects, Unix has ulimits- relevant: Linux resource limitation) and then keep things like os.execute, half the io-library and modules like luasocket away from the scripter (pretty easy).

Recovering from errors in sandboxed code

You can handle almost everything (except violation of time/memory limits) without trashing your Lua interpreter: Just wrap the execution of user-supplied code inside a pcall; if you call any Lua-API functions that might fail yourself, you need to wrap them inside a function that you can pcall, too (or set a Lua panic function and handle it from there).

---

[I didn't want people glancing at this thread to assume that debug.sethook is adequate for sandboxing, and stackoverflow would not let me comment (yet)]

Answer 2

Put the Lua loop inside a coroutine and at the end of every loop, just use Yield return null to wait for a frame (Allowing the worker thread to be executed).

Answer 3

I suggest handling it just like any other Lua error. In other words, threat the code above as if the user had just written

error("Script execution has been cancelled after stalling for too long")

(I'm assuming that you detect infinite loops by assuming that no script should take more than a fixed amount of time to run. Change the message appropiatedly if this is not the case)

The way you will have to handle this will depend on how you deal with Lua errors - but you will have to do it anyway, so chances are that most of the code is there already.

EDIT: It seems that Martin James' suggestion is your best option. You can use the debug.setHook() to run some lua code every 100 Lua instructions or so, and if too much time has passed on the same client code, throw the error. Details about this can be found on this mailing list, and code sample on the lua-project repo.

Answer 4

I have used two techniques to address this issue. The first is to call the DoFile or DoString method inside a separate Task. That way you can abort the thread. I do not know of any technique (say from within a hook) to exit the interpreter.

Another way would be to setup a separate Appdomain. Using this you can also set separate constraints (evidence and permission set) see CreateDomain for details.

Answer 5

What about setting line hook before running lua code? I don't know is it possible in C#, but it is possible in original C lua.

Is yor loop detector already run in different C# task?

• If yes, use C# variable with C# hook. In this case your loop detector set and hool check C# variable like terminateLua and throw lua error (it should be possible).
• If no, and you detect loop on lua side (also using hooks?), set lua variable (make it upvalue to prevent user tricks) and also throw error.

Important part is to don't reset this (either C# or lua) variable before you are really sure about finishing your task, because user can catch your error via pcall and try to handle it.

Answer 6

You can do it like this, if you want to detect if a loop is still running.

local detect = false

detect == true

goal = 5
star = 0

function loop()
   while detect == true do
       print("Still running")
   else
       print("Loop stopped running")
   end
end

Spawn(loop)

while true do
   if star > goal then
      detect = false
   end
end

The output will be:

Still running
Still running
Still running
Still running
Still running
Loop stopped running