Why does my tail calling function not pause and flush output?

Question

I have looked through lua-users Sleep Function reference in an effort to find a non-busy waiting solution to the sleep problem and I'm not happy with any of them. Nonetheless I attempted using several to provide a delay at the end of a function that uses tail calls.

Normally I wouldn't use tail calls but since lua doesn't leave a stack behind for tail calls it suits me.

Unfortunately I see my processor spike to about 20% use and the program becomes immediately unresponsive without any flushing of output once it begins.

The problem (simplified) looks like this:

function myFunc ()
   -- do some stuff
   -- lots of snazzy logic and function calls
   -- heck, throw in a few prints
   print "Going to sleep"
   -- sleep for a bit
   os.execute("sleep 10")
   print "Waking up"
   -- tail call
   return myFunc()
end

I have tried the socket select method, os.execute and of course busy waiting. Of these only busy waiting gives the expected behaviour.

Are these other non-busy waiting solutions also non blocking? That is do they allow the tail call to be processed despite the delays?

How can I flush the output and have the function wait 10 seconds before resuming without busy waiting?

That's not a tail call. You need `return myfunc()` for that. — Etan Reisner, Oct 16 '15 at 03:31
I'm not sure I understand the goal here exactly. What's the goal here? What's the problem? — Etan Reisner, Oct 16 '15 at 03:32
@EtanReisner It is a control loop for a script that runs 24-7. — Stephen, Oct 16 '15 at 11:42
As an update, when I corrected the tail call I still observed identical behaviour. There are other ways I can accomplish my goal however this one guarantees I don't get overlapping execution where calling the function from an external timer does not, unless I provide a guard mechanism (which is not at all unreasonable) — Stephen, Oct 16 '15 at 12:48
What is the incorrect behavior here? What is happening that shouldn't be or isn't happening that should be? Also how does sleeping ensure you don't overlap? Either the jobs run until they finish before you continue so they can't overlap or they run in the background and a sleep is just a guess as to how long they will take but that guess can be wrong in which case you can overlap. — Etan Reisner, Oct 16 '15 at 15:25
What is happening, that should not be, is that I never see any of the print output and the CPU use spikes as long as the script runs, the rest of the software becomes unresponsive. What should happen is that the sleep sections should be releasing CPU use so that the rest of the system can progress. The reason this guarantees no overlap is that the rest of the algorithm completes before the sleep statement, the sleep happens and then we wake up and make the tail call. — Stephen, Oct 16 '15 at 17:10
You see that with that *exact* function? Called from where/what context? The standalone interpreter? Output to console/screen? Piped to something? In an embedded application? And the sleep doesn't guarantee the no overlap there. The process finishing in the foreground does. The sleep just lets the OS schedule other tasks for a bit without the pressure of the process running (but we are splitting unimportant hairs at this point). — Etan Reisner, Oct 16 '15 at 18:30
The context is that the script is called one time, only when a particular character connects by telnet to the Realms of Despair using MushClient as a client. The exact function is about 200 lines long and is simply a state machine that can call other functions. This delay of 10 seconds appears at the end of the complete loop of state executions. None of the function calls are threaded therefore from start to finish the real version of myFunc() runs sequentially reaching the sleep() call after all subroutines have completed. — Stephen, Oct 16 '15 at 20:38
The tail call executes after the sleep call finishes (the sleep is blocking this loop from running any further sequential instructions) and the next iteration is called last. Thus there are never 2 copies of myFunc() running anywhere at any time. Or do you see something in my reasoning that's amiss? I can call an alarm and fire a copy of myFunc() without the tail call every n seconds, with n being arbitrary, but if for some reason a subroutine gets hung up waiting for something I will then risk having multiple versions of myFunc() live simultaneously which will corrupt the state. — Stephen, Oct 16 '15 at 20:40
The tail-call *already* ensures that your function will never be running at the same time twice because you only call it once and then it calls itself only when it is finished. The sleep has nothing to do with that fact. (But we really aren't getting anywhere with this part of things so lets just ignore it at least for now.) Where should that `print` output be showing up? Does `print` in other code in other scripts work correctly? Does `print` in other places in that code (or the other functions) print correctly? Is there some other logging function you should be using instead? — Etan Reisner, Oct 16 '15 at 21:12
I agree, the tail-call is what is ensuring a singleton approach. I was only explaining why using an external timer was not a desirable approach. The print output would display in a terminal window, and if I remove the tail call it appears correctly. What I believe is happening is that the sleep is not occurring which puts the whole thing into a tight loop. Could lua be optimizing the statement away? — Stephen, Oct 17 '15 at 17:47
What do you mean "if I remove the tail call"? I assume you don't mean if you just go back to your original version which doesn't include `return` and so isn't actually a tail-call? I assume you mean some other version of the function? If that's the case **what** other version of the code prints correctly? Are you sure your code is, in fact, hitting the tail-call and looping and not getting stuck in some other location? (No, lua is not optimizing that call away. It is possible the call to `sleep` is *failing* in which case it will not wait and will tight-loop instead.) — Etan Reisner, Oct 18 '15 at 19:27
By "remove the tail call" I mean commenting it out and observe the execution where I observe the print, the pause and the subsequent output. Placing the tail call back in then returns to the tight loop like behaviour. — Stephen, Oct 19 '15 at 01:25
You see the output in real-time as the script executes that way? That's interesting. I don't see how the presence of the tail-call could affect that in any way (unless the embedding environment is doing very complex/clever things). Without the tail call your code should absolutely eventually run out of stack frames and explode (again unless the embedding environment is going something complicated and clever like stopping and restarting the script with saved state or something). — Etan Reisner, Oct 19 '15 at 01:39
Agreed. It has also been pointed out that a while loop will suit my purposes wonderfully, making me feel rather silly on this whole thing. — Stephen, Oct 19 '15 at 01:41

Stephen · Accepted Answer · 2015-10-19T01:52:43.833

On the advice of Nick Gammon I tried his wait.lua solution. My initial attempt:

function controlLoop()
   wait.make (
      function()
         world.Note("Hello world.") -- essentially print
        wait.time(10)
     end
   )
   world.Note("Goodbye world.") -- essentially print
  return controlLoop()
end

Suffered from precisely the same 100% CPU use, no output displayed behaviour.

My second attempt:

function controlLoop()
   wait.make (
      function()
         world.Note("Hello world.")
         wait.time(10)
         world.Note("Goodbye world.")
         return controlLoop()
      end
   )
end

Has been running for 3 hours without fault. I did put a debug call to the stack trace using debug.traceback() and never got a response more than 1 level deep. Additionally watching the Window memory use for the process, it did not increase over the 3 hours.

I am pleased that I have a solution but I am still somewhat displeased that I don't understand why it is working and why the original version fails.

It has been pointed out to me that I'm suffering from tunnel vision and that a while loop will solve my problem famously.

function controlLoop()
   wait.make (
      function()
         while true do
            world.Note("Hello world.")
            wait.time(10)
            world.Note("Goodbye world.")
         end -- loop
      end
   )
end

To which I can only reply ... duh, of course.

Why does my tail calling function not pause and flush output?

1 Answers1