How to deal with a child thread that hangs?

Question

I have a process to extract data from an API that uses multiple threads that is hanging.

• The main thread launches the child threads and waits until N numbers of API calls have been made and all child threads have ended to finish;
• 1 child thread populates a queue with calls that need to be made to an API;
• 8 child threads execute the API calls.

When one of the API call hangs (and I can't control the timeout, unfortunately), the child thread never finishes and the main thread will continue forever waiting for the child thread to end.

Is there any way to either force the child threads to end from the main thread? Alternatively is there a tried and tested way to do this type of data collection process that does not generate this issue?

Answer 1

When a thread is created, you can control whether it will be automatically terminated when the main thread quits by explicitly setting its daemon attribute before calling its start() method. See the Thread Objects section of the online docs.

There's also a fairly detailed explanation in my answer to a similar question:
       What does sys.exit really do with multiple threads?.