3

I am making my existing .Net Application Scriptable for non programming users. I added lua, it works like a charm. Then I added debug functionality(pause/continue/step) via debug.sethook. It works also like a charm.

Now I realize that my Application needs edit and continue feature like Visual Studio has it. You pause the execution can edit the code and then continue from your current state with changes applied. This feature is very important to me. I thought this would be easy to do for scripting languages.

Everywhere I read that scripting languages can do this. But even after spending hours of searching I haven't found an Lua implementation yet. It hasn't to be Lua but hot swapping code in Lua would be my first choice.

How can the ability for the user be offered to pause and edit the script and than continue the execution with changes applied?

NOTE: It doesn't have to be Lua every scripting language would be okay

Update

@Schollii Here is an example:

function doOnX() 
   if getValue() == "200" then
      value = getCalculation()
      doSomething() -- many function calls, each can take about 2s
      doSomething()
      doSomething()
      print(value)
      doX(value)
   end
end

doOnX()

Thank you for your suggestions. This is how it might work:

  1. I will use https://github.com/frabert/NetLua Its a very cool, well written 100% C# Lua Interpreter. It is generating an AST tree first and then directly executing it.
  2. The parser needs to be modified. In Parser.cs public Ast.Block ParseString(string Chunk) there is a parseTree generated first. parseTree.tokens[i].locations are containing the exact position of each token. The Irony.Parsing.ParseTree is then parsed again and is converted to a NetLua.Ast.Block but the location information is missed. I will need to change that so later I will know which Statement is in which line.
  3. Now each Statement from the AST tree is directly executed via EvalBlock. Debug functionality (like I have in my C Binding lua Interpreter DynamicLua via debug.setHook) needs to be added. This can be done in LuaInterpreter.cs internal static LuaArguments EvalBlock(`. Pause/continue/step functions should be no problem. I also can now add current line Highlighting because each statement contains position line information.
  4. When the execution is paused and the code was edited the current LuaContxct is saved. It contains all variables. Also the last Statement with the last execution line is saved.
  5. Now the code String is parsed again to a new AST tree. It gets executed. But all statements are skipped until the saved statement with the line statement is reached. The saved LuaContext is restored and execution can continue with all changes applied.

New variables could be also added after the last executed line, because a new NetLua.Ast.Assignment Statement could just add a new variable to the current LuaContext and everything should just work fine.

Will this work?

Community
  • 1
  • 1
pixelport
  • 83
  • 1
  • 6
  • 1
    Just to be clear: is it enough to let users swap in new scripts in your application without restarting the app? Or do you actually need to let users somehow edit their scripts whilst execution is suspended inside their code? If the latter, what would you want to happen if resuming execution is fundamentally impossible, e.g. if the edits have removed the function being executed or so? Some clarification about the limitations you're willing to live with would be useful. – John Zwinck Oct 04 '14 at 11:03
  • I am open for everything. It would also be okay to only add code after the current executed line but with all variables values still be available. Of course there are limitations like new added functions or, table structure edits. But most of the time the script only consists of function calls and setting and accessing a few variables and some if statements. Visual Studio also cant handle all edits. For example you cant change Class field edits or edits in anonymous functions. But the feature is totally awesome. Please tell all your thoughts and possibilitys. Thank you for contributing! – pixelport Oct 04 '14 at 11:49
  • 1
    I think the first step might be to get the basics working. You could try to get the point where the user can edit their script and save it, and you could auto-detect the script file changing, tear down the old Lua state and load it again. It's not really edit-and-continue at the line-of-code level, but it'd be something to start with. – John Zwinck Oct 04 '14 at 13:08
  • Saving the variables using _G is doable. The Problem with that is how do you save the execution state. You could shrink the lua string and execute it but what if the user pauses inside a if statement? The lua interpreter will crash because of an unexpected end statement. – pixelport Oct 04 '14 at 13:11
  • I would say don't try to save the execution state to the level of being able to pause and resume within a user's own code. Just let their function run to completion, reload the Lua completely, and let the next call into their code start with the new code. Baby steps. :) – John Zwinck Oct 04 '14 at 13:20
  • How would you do "all into their code start with the new code"? This is a good idea and it would definitely be possible. But that would be it. The user could not edit inside if statements or functions. I could parse the lua code to add dummy if statements on the beginning to make it work. – pixelport Oct 04 '14 at 13:25
  • 1
    Another idea is to record every function call you make into the user's script--what function you called and with what arguments--so that after reloading their code you can make the exact same series of calls again from the beginning. A sort of replay mode if you will. I'm afraid this is starting to get a bit chatty though, so we should try to wrap up the discussion in these comments. :) I suggest trying out some of the ideas here and coming back with more concrete implementation questions. – John Zwinck Oct 04 '14 at 13:28
  • F# is closer to Lua than C#. In Visual Studio, [Edit and Continue Not Supported for F#](http://msdn.microsoft.com/en-us/library/ee843933.aspx). – Tom Blodget Oct 04 '14 at 22:45
  • @TomBlodget Interesting thought. But it seems that it is more a priority issue by Microsoft. http://stackoverflow.com/questions/24233445/why-doesnt-f-support-edit-and-continue – pixelport Oct 04 '14 at 22:53
  • 1
    IIRC: A debugger I wrote allowed conditional breakpoint expressions to be added. That would be similar to adding a line to a function. We bound variable references to locals (inc. parameters), existing upvalues in the current stack frame and globals (last resort). New upvalues could not be created. – Tom Blodget Oct 04 '14 at 23:42

1 Answers1

2

I think this is quite challenging and triicky to do right.

Probably the only way you could do that is if you recompile the chunk of code completely. In a function this would mean the whole function regardless of where edit is in function. Then call the function again. Clearly the function must be re-entrant else its side effects (like having incremented a global or upvalue) would have to be undone which isn't possible. If it is not reentrant it will still work just not give expected results (for example if the function increments a global variable by 1 calling it again will result in the global variable having been increased by 2 once the function finally returns).

But finding the lines in the script where the chunknstarts and ends would be tricky if truly generic solution. For specific solution you would have to post specific examples of scripts you want to run and examples of lines you would want to edit. If the whole user script gets recompiled and rerun then this is not a problem, but the side effects is still an issue, examples could help there too.

Oliver
  • 27,510
  • 9
  • 72
  • 103
  • I updated my question with more details for you. To recompile the code completely is a good idea. I have made a plan how this could be implemented. Please check it out. – pixelport Oct 05 '14 at 13:25
  • Yes I think what you propose in your edit should work. – Oliver Oct 05 '14 at 15:13