How can I end an infinite loop with socket operations inside after finishing current iteration?

Question

I have an infinite loop in which there are operations that are mandatory to be completely executed before exiting the loop. Namely, I am using the socket library for connecting to an external device and I need to wait the read instructions to be finished before interrupting the loop.

I have tried using a signal handler (like in this question) for raising a flag when a Keyboard interrupt is detected.

Current code:

import videosensor
import signal

def signal_handler(signal, frame):
    """Raises a flag when a keyboard interrupt is raised."""
    global interrupted
    interrupted = True

if __name__ == '__main__':
    camera = videosensor.VideoSensor(filename)
    interrupted = False
    signal.signal(signal.SIGINT, signal_handler)

    while not interrupted:
        location = camera.get_register()
        #...
        #More irrelevant stuff is executed.
        #...
        time.sleep(0.01)

    #This code has to be executed after exiting while loop
    camera_shutdown(camera)

In the previous code, videosensor.VideoSensor is a class containing socket operations for getting data from an external device. The get_register() method used in the main routine is the following:

def get_register(self):
    """Read the content of the specified register.
    """
    #Do some stuff
    value = socket.recv(2048)
    return value

The problem:

I wanted the while loop to be continually executed until the user pressed a key or used the Keyboard Interrupt, but after the current iteration was finished. Instead, using the previous solution does not work as desired, as it interrupts the ongoing instruction, and if it is reading the socket, an error is raised:

/home/.../client.pyc in read_register(self, regkey)

    164         reg = self._REGISTERS[regkey]
    165         self.send('r,{}\n'.format(reg))
--> 166         value = socket.recv(2048)
    167         #Convert the string input into a valid value e.g. list or int
    168         formatted_result = ast.literal_eval(value)

error: [Errno 4] Interrupted system

EDIT: It seems, from an answer below, that there is no way of using the Keyboard Interrupt and avoid the socket read function to be aborted. Despite there are solutions for catching the error, they don't avoid the read cancellation.

I am interested, though, in finding a way of getting a user input e.g. specific key press, that raises the flag, which will be checked at the end of the loop, without interrupting the main routine execution until this check.

EDIT2: The used OS is the Linux distribution Ubuntu 14.04

Possible duplicate of [Catching / blocking SIGINT during system call](http://stackoverflow.com/questions/3016369/catching-blocking-sigint-during-system-call) — zmo, Dec 07 '16 at 14:31
@zmo I edited the content specifying why that solution is not suitable. Anyway, thanks for the proposal. — Jalo, Dec 07 '16 at 14:49
What platform are you running your code on - Windows or Unix? — mhawke, Dec 07 '16 at 21:11
@Jalo: OK, take a look at my answer and see if it works for you. — mhawke, Dec 09 '16 at 08:59

score 1 · Accepted Answer · edited May 23 '17 at 12:26

1

After quick SO search I found this solution for your issue

Basically, there's nothing you can do: when you send a SIGINT to your process, the socket will return a SIGINT as well. The best you can do, then, is to actively ignore the issue, by catching the socket EINTR error and going on with your loop:

import errno

try:
    # do something
    value = conn.recv(2048)
except socket.error as (code, msg):
    if code != errno.EINTR:
        raise

An alternative solution to avoid issues with C-c breaking reads, is to use parallel execution, to read your socket in a routine, and handle user input on the other:

import asyncio

async def camera_task(has_ended, filename):
    camera = videosensor.VideoSensor(filename)

    try:
        while not has_ended.is_set():
            location = camera.get_register()
            #...
            #More irrelevant stuff is executed.
            #...
            await asyncio.sleep(0.01)
    finally:
        #This code has to be executed after exiting while loop
        camera_shutdown(camera)

async def input_task(shall_end):
    while True:
        i = input("Press 'q' to stop the script…")
        if i == 'q':
            shall_end.set()

def main():
    filename = …
    #
    end_event = asyncio.Event()
    asyncio.Task(camera_task(end_event, filename))
    asyncio.Task(input_task(end_event))
    asyncio.get_event_loop().run_forever()

or with threading

import threading, time

def camera_task(has_ended, filename):
    camera = videosensor.VideoSensor(filename)

    try:
        while not has_ended.is_set():
            location = camera.get_register()
            #...
            #More irrelevant stuff is executed.
            #...
            time.sleep(0.01)
    finally:
        #This code has to be executed after exiting while loop
        camera_shutdown(camera)

def input_task(shall_end):
    while True:
        i = input("Press 'q' to stop the script…")
        if i == 'q':
            shall_end.set()

def main():
    filename = …
    #
    end_event = threading.Event()
    threads = [
        threading.Thread(target=camera_task, args=(end_event, filename)),
        threading.Thread(target=input_task, args=(end_event,))
    ]
    # start threads
    for thread in threads:
        thread.start()
    # wait for them to end
    for thread in threads:
        thread.join()

or with multiprocessing:

import multiprocessing, time

def camera_task(has_ended, filename):
    camera = videosensor.VideoSensor(filename)

    try:
        while not has_ended.is_set():
            location = camera.get_register()
            #...
            #More irrelevant stuff is executed.
            #...
            time.sleep(0.01)
    finally:
        #This code has to be executed after exiting while loop
        camera_shutdown(camera)

def input_task(shall_end):
    while True:
        i = input("Press 'q' to stop the script…")
        if i == 'q':
            shall_end.set()

def main():
    filename = …
    #
    end_event = multiprocessing.Event()
    processes = [
        multiprocessing.Process(target=camera_task, args=(end_event, filename)),
        multiprocessing.Process(target=input_task, args=(end_event,))
    ]
    # start processes
    for process in processes:
        process.start()
    # wait for them to end
    for process in processes:
        process.join()

disclaimer: those codes are untested, and there might be some typos or little errors, but I believe the overall logic should be

edited May 23 '17 at 12:26

Community

1
1

answered Dec 07 '16 at 14:34

zmo

24,463
4
54
90

Thank you for your proposal. However, my main problem is to avoid socket.recv() to stop receiving data if user requests to interrupt. With the solution you propose, the error would be ignored, but the recv() call will be aborted as well... My question was more oriented to finding an alternative way of getting a user input, not necessarily a Keyboard Interrupt, that raises the flag which will be checked at the end of the loop. – Jalo Dec 07 '16 at 14:44
well, I might be wrong, so please check with real code, but I'd except that you're not loosing data upon `EINTR`, the buffered data will be returned with the following call, which is why ignoring the `EINTR` exception should work for you when hitting `C-c`. Otherwise, your solution is to put your socket stuff either in a thread, a forked process or an async coroutine, and have another one handling CLI user input (with a little prompt saying: `press 'q' to stop…` – zmo Dec 07 '16 at 14:51
I updated with a solution using `asyncio`. It's not involving the complexity and risks of threads and it's not having the cost of processes. It's only code that will run starting with recent py3. – zmo Dec 07 '16 at 15:07
I am sorry, but my code is in for Python 2.7 and it seems that the asyncio library is only availablo in Python3. Is there a similar solution available for Python2? – Jalo Dec 07 '16 at 16:34
The first option is still not valid for me. It seems that when interrupted, the connection gets broken and any data gets stored. I get an error in the next line of code that uses 'value' saying **UnboundLocalError: local variable 'result' referenced before assignment** – Jalo Dec 07 '16 at 16:42
My best advice for you is to switch to py3, py2 is now end of life, the future lies in py3! If really you want py2, you can use [trollius](https://pypi.python.org/pypi/trollius/) that is an implementation of asyncio in py2, or twisted, or greenlets, or tornado… – zmo Dec 07 '16 at 16:43
*It seems that when interrupted* well, there's not much you can do against that. You cannot make your program ignore signals, because signals are designed to interfere with your program. For example, if you do `C-\` your program will get killed directly, and there's nothing you can do against that. What you can, though, is to offer a way for the user to exit the program nicely with a keystroke, and not abruptly with a signal. – zmo Dec 07 '16 at 16:48
I am working on a big project and it is not possible to switch to py3, though I intend to use it in future projects... I have never used trollius, I will take a look to it and try to use it for my problem. Anyway, would it be very hard for you to adapt your answer to py2? I am also considering using a multithreading solution similar to the other answer – Jalo Dec 07 '16 at 16:48
well, the same principle I presented above can work with the threading module and the multiprocess module. – zmo Dec 07 '16 at 17:10
just adding multiprocessing and threading versions of the same. – zmo Dec 07 '16 at 17:23
Thank you so much for your help and your time. I finally used the multithreading option and it works fine. Just 2 issues: the while loopin the input_task thread never ends. It just needs a small correction, but nothing too hard. The second issue is that I did no use the `try - finally` statements. I am intrigued by why you used it, I would really appreciate an explanation. Anyway, it was a great and complete answer! – Jalo Dec 09 '16 at 10:05
I've added the `try/finally` so you can gracefully disconnect from the device no matter what, even upon failure in the `try` block (whether this is a `C-c` or `socket` failure or something else). I cannot now whether the `finally` block is right — that's up to you — but I thought having it can be helpful to you. – zmo Dec 09 '16 at 10:10

J Darbyshire · Answer 2 · 2016-12-07T14:18:27.833

0

You created your custom signal handler but did not overide the default keyboard interrupt behaviour. Add signal.signal(signal.SIGINT, signal_handler) to your code to accomplish this:

import videosensor
import signal

# Custom signal handler
def signal_handler(signal, frame):
    """Raises a flag when a keyboard interrupt is raised."""
    global interrupted
    interrupted = True

# Necessary to override default keyboard interrupt
signal.signal(signal.SIGINT, signal_handler)  

if __name__ == '__main__':
    # Main programme

edited Dec 07 '16 at 14:18

answered Dec 07 '16 at 14:07

J Darbyshire

382
1
7

I am sorry, I actually wrote that instruction in my code, but I forgot to put it in the OP. I already edited it. That is not the problem, then... Thank you anyway for your answer. – Jalo Dec 07 '16 at 14:25

mhawke · Answer 3 · 2016-12-08T21:56:37.173

If I understand correctly, you do not want socket.recv() to be interrupted, but you do want to use signals to let the user indicate that the I/O loop should be terminated once the current I/O operation has completed.

With the assumption that you are using Python 2 on a Unix system, you can solve your problem by calling signal.siginterrupt(signal.SIGINT, False) before entering the loop. This will cause system calls to be restarted when a signal occurs rather than interrupting it and raising an exception.

In your case this means that the socket.recv() operation will be restarted after your signal handler is called and therefore get_register() will not return until a message is received on the socket. If that is what you want your code will be:

    interrupted = False
    old_handler = signal.signal(signal.SIGINT, signal_handler)    # install signal handler
    signal.siginterrupt(signal.SIGINT, False)                     # do not interrupt system calls

    while not interrupted:
        location = camera.get_register()
        if location == '':
            # remote connection closed
            break
        #...
        #More irrelevant stuff is executed.
        #...
        time.sleep(0.01)

That's one way to do it, but it does require that your code is running on a Unix platform.

Another way, which might work on other platforms, is to handle the exception, ignore further SIGINT signals (in case the user hits interrupt again), and then perform a final socket.recv() before returning from the get_register() function:

import errno

def get_register(s):
    """Read the content of the specified register.
    """
    #Do some stuff
    try:
        old_handler = None
        return s.recv(2048)
    except socket.error as exc:
        if exc.errno == errno.EINTR:
            old_handler = signal.signal(signal.SIGINT, signal.SIG_IGN)    # ignore this signal
            return s.recv(2048)    # system call was interrupted, restart it
        else:
            raise
    finally:
        if old_handler is not None:
            signal.signal(signal.SIGINT, old_handler)    # restore handler

Signal handling can get tricky and there might be race conditions in the above that I am not aware of. Try to use siginterrupt() if possible.

I tried it, and it fails about 20% of times. Like in the solution proposed below, it raises the error in the next line of code that uses 'value' saying **UnboundLocalError: local variable 'result' referenced before assignment**. I guess that when the interrupt does not coincide at the same time as the recv instruction is when the code works. — Jalo, Dec 09 '16 at 09:41
Anyway, did you wrote the `if` statement for any special reason? — Jalo, Dec 09 '16 at 09:42
@Jalo: regarding the error that you see, which of the 2 solutions that I show are you using? `siginterrupt()` should work fine. I also just noticed that you also are calling a `send()` function prior to the `recv()` - if using the second solutoin you need to make sure that that is also safe from signals. Finally, accessing unbound variables is more likely a bug in your code wherein you assume that a variable is bound to some value but an exception prevents that from occuring. In the code that I show the value is returned directly without using any variables. — mhawke, Dec 09 '16 at 12:48
I tried the first solution, and I refer to the `if location='':` statement. The get_register method first sends a value indicating the register it wants to read, and the reads the content that the client should send back. Regarding the unbound variable, _result_ is the same variable as _value_, but I made a transcript mistake when writing the message here. — Jalo, Dec 09 '16 at 13:13
@Jalo: `if location == ''` checks whether the server/peer has closed the connection (`socket.recv()` returns an empty string in that case.) — mhawke, Dec 09 '16 at 22:45

How can I end an infinite loop with socket operations inside after finishing current iteration?

Current code:

The problem:

3 Answers3