Interact with long running python process

Question

I have a long running python process running headless on a raspberrypi (controlling a garden) like so:

from time import sleep

def run_garden():
    while 1:
        /* do work */
        sleep(60)

if __name__ == "__main__":
    run_garden()

The 60 second sleep period is plenty of time for any changes happening in my garden (humidity, air temp, turn on pump, turn off fan etc), BUT what if i want to manually override these things?

Currently, in my /* do work */ loop, i first call out to another server where I keep config variables, and I can update those config variables via a web console, but it lacks any sort of real time feel, because it relies on the 60 second loop (e.g. you might update the web console, and then wait 45 seconds for the desired effect to take effect)

The raspberryPi running run_garden() is dedicated to the garden and it is basically the only thing taking up resources. So i know i have room to do something, I just dont know what.

Once the loop picks up the fact that a config var has been updated, the loop could then do exponential backoff to keep checking for interaction, rather than wait 60 seconds, but it just doesnt feel like that is a whole lot better.

Is there a better way to basically jump into this long running process?

score 1 · Answer 1 · edited May 23 '17 at 10:28

1

Why not use an event based loop instead of sleeping for a certain amount of time.

That way your loop will only run when a change is detected, and it will always run when a change is detected (which is the point of your question?).

You can do such a thing by using:
python event objects

Just wait for one or all of your event objects to be triggered and run the loop. You can also wait for X events to be done, etc, depending if you expect one variable to be updated a lot.

Or even a system like: broadcasting events

edited May 23 '17 at 10:28

Community

1
1

answered Mar 24 '13 at 13:25

Serdalis

10,296
2
38
58

this would have worked, but I ended up using a simple web server with a timeout to kill two birds with one stone – MattoTodd Mar 25 '13 at 01:37

score 1 · Accepted Answer · answered Mar 24 '13 at 13:34

Listen on a socket in your main loop. Use a timeout (e.g. of 60 seconds, the time until the next garden update should be performed) on your socket read calls so you get back to your normal functionality at least every minute when there are no commands coming in.

If you need garden-tending updates to happen no faster than every minute you need to check the time since the last update, since read calls will complete significantly faster when there are commands coming in.

jedwards · Answer 3 · 2013-03-24T13:53:02.187

Python's select module sounds like it might be helpful.

If you've ever used the unix analog (for example in socket programming maybe?), then it'll be familiar.

If not, here is the select section of a C sockets reference I often recommend. And here is what looks like a nice writeup of the module.

Warning: the first reference is specifically about C, not Python, but the concept of the select system call is the same, so the discussion might be helpful.

Basically, it allows you to tell it what events you're interested in (for example, socket data arrival, keyboard event), and it'll block either forever, or until a timeout you specify elapses.

If you're using sockets, then adding the socket and stdin to the list of events you're interested in is easy. If you're just looking for a way to "conditionally sleep" for 60 seconds unless/until a keypress is detected, this would work just as well.

EDIT:

Another way to solve this would be to have your raspberry-pi "register" with the server running the web console. This could involve a little bit extra work, but it would give you the realtime effect you're looking for.

Basically, the raspberry-pi "registers" itself, by alerting the server about itself, and the server stores the address of the device. If using TCP, you could keep a connection open (which might be important if you have firewalls to deal with). If using UDP you could bind the port on the device before registering, allowing the server to respond to the source address of the "announcement".

Once announced, when config. options change on the server, one of two things usually happen:

A) You send a tiny "ping" (in the general sense, not the ICMP host detection protocol) to the device alerting it that config options have changed. At this point the host would immediately request the full config. set, acquiring the update with it.

B) You send the updated config. option (or maybe the entire config. set) back to the device. This decreases the number of messages between the device and server, but would probably take more work as it seems like more a deviation from your current setup.

If using option A in your edit...would the ping hit a second process listening, or the original garden process (which would now be listening)? — MattoTodd, Mar 24 '13 at 14:01
Well you could do it either way, but I'd do the latter. Refactor the original process to first register, then wait for a ping (probably via a blocking socket call). Once the ping is received, the script continues at that point and fetches the config. set. — jedwards, Mar 24 '13 at 14:16
if i am blocking by waiting for the ping, then how does my garden continue to run its loop every 60 seconds? am i missing something? — MattoTodd, Mar 24 '13 at 14:23
No, I was. In that case I'd either go multithreaded or simply [set a 60 second timeout](http://docs.python.org/2/library/socket.html#socket.socket.settimeout). If you go the timeout route, the only additional delay you'd get would be if the ping was received while doing your main loop tasks. In this case, the ping would be buffered on the network stack and immediately handled as soon as your main loop tasks completed. — jedwards, Mar 24 '13 at 14:28

Interact with long running python process

3 Answers3