Safe way to exit an infinite loop within a Thread Pool for Python3

Question

I am using Python3 modules:

• requests for HTTP GET calls to a few Particle Photons which are set up as simple HTTP Servers

• As a client I am using the Raspberry Pi (which is also an Access Point) as a HTTP Client which uses multiprocessing.dummy.Pool for making HTTP GET resquests to the above mentioned photons

The polling routine is as follows:

def pollURL(url_of_photon):
    """
    pollURL: Obtain the IP Address and create a URL for HTTP GET Request
    @param: url_of_photon: IP address of the Photon connected to A.P.
    """
    create_request = 'http://' + url_of_photon + ':80'
    while True:
        try:
            time.sleep(0.1) # poll every 100ms
            response = requests.get(create_request)
            if response.status_code == 200:
                # if success then dump the data into a temp dump file
                with open('temp_data_dump', 'a+') as jFile:
                    json.dump(response.json(), jFile)
            else:
               # Currently just break
               break
        except KeyboardInterrupt as e:
            print('KeyboardInterrupt detected ', e)
            break

The url_of_photon values are simple IPv4 Addresses obtained from the dnsmasq.leases file available on the Pi.

the main() function:

def main():
    # obtain the IP and MAC addresses from the Lease file
    IP_addresses = []
    MAC_addresses = []
    with open('/var/lib/misc/dnsmasq.leases', 'r') as leases_file:
        # split lines and words to obtain the useful stuff.
        for lines in leases_file:
            fields = lines.strip().split()
            # use logging in future
            print('Photon with MAC: %s has IP address: %s' %(fields[1],fields[2]))
            IP_addresses.append(fields[2])
            MAC_addresses.append(fields[1])

            # Create Thread Pool
            pool = ThreadPool(len(IP_addresses))
            results = pool.map(pollURL, IP_addresses)
            pool.close()
            pool.join()

if __name__ == '__main__':
    main()

Problem

The program runs well however when I press CTRL + C the program does not terminate. Upon digging I found that the way to do so is using CTRL + \

How do I use this in my pollURL function for a safe way to exit the program, i.e. perform poll.join() so no leftover processes are left?

notes:

the KeyboardInterrupt is never recognized with the function. Hence I am facing trouble trying to detect CTRL + \.

Answer 1

The pollURL is executed in another thread. In Python, signals are handled only in the main thread. Therefore, SIGINT will raise the KeyboardInterrupt only in the main thread.

From the signal documentation:

Signals and threads

Python signal handlers are always executed in the main Python thread, even if the signal was received in another thread. This means that signals can’t be used as a means of inter-thread communication. You can use the synchronization primitives from the threading module instead.

Besides, only the main thread is allowed to set a new signal handler.

You can implement your solution in the following way (pseudocode).

event = threading.Event()

def looping_function( ... ):
    while event.is_set():
        do_your_stuff()

def main():
    try:
        event.set()
        pool = ThreadPool()
        pool.map( ... )
    except KeyboardInterrupt:
        event.clear()
    finally:
        pool.close()
        pool.join()