0

This is a two part question,

  1. After I cancel my script it still continues run, what I'm doing is queering an exchange api and saving the data for various assets.

My parent script can be seen here you can see i'm testing it out with just 3 assets, a sample of one of the child scripts can be seen here.

After I cancel the script the script for BTC seems to still be running and new .json files are still being generated in it's respective folder. The only way to stop it is to delete the folder and create it again.

  1. This is really a bonus, my code was working with two assets but now with the addition of another it seems to only take in data for BTC and not the other 2.
David Hancock
  • 1,063
  • 4
  • 16
  • 28
  • 1
    Have you looked at catching Keyboard interrupt http://stackoverflow.com/questions/21120947/catching-keyboardinterrupt-in-python-during-program-shutdown – gogasca Feb 21 '16 at 17:52
  • We prefer scripts (that have been pared down to a small workable example) to be posted directly in the question. Keep it all self contained. – tdelaney Feb 21 '16 at 18:10
  • @spicyramen thanks, I think that might have worked. btw I'm in japan at the moment and am in love with the spicy ramen. – David Hancock Feb 21 '16 at 18:25
  • What do you mean by "cancel the script"? Are you running from the command line and hitting ctrl-c? – tdelaney Feb 21 '16 at 18:33
  • I can't explain why it doesn't cancel, but you have a bigger problem. Notice in `t1 = Thread(target=BTC.main())` that you called `main` instead of just referenced it. Your program will execute main and use its return value to try to create a thread. You should do `t1 = Thread(target=BTC.main)`.... and I think that answers your bonus question. – tdelaney Feb 21 '16 at 18:35

1 Answers1

0

Your first problem is that you are not really creating worker threads. t1 = Thread(target=BTC.main()) executes BTC.main() and uses its return code to try to start a thread. Since main loops forever, you don't start any other threads.

Once you fix that, you'll still have a problem.

In python, only the root thread sees signals such as ctrl-c. Other threads will continue executing no matter how hard you press the key. When python exits, it tries to join non-daemon threads and that can cause the program to hang. The main thread is waiting for a thread to terminate, but the thread is happily continuing with its execution.

You seem to be depending on this in your code. Your parent starts a bunch of threads (or will, when you fix the first bug) and then exits. Really, its waiting for the threads to exit. If you solve the problem with daemon threads (below), you'll also need to add code for your thread to wait and not exit.

Back to the thread problem...

One solution is to mark threads as "daemon" (do mythread.daemon = True before starting the thread). Python won't wait for those threads and the threads will be killed when the main thread exits. This is great if you don't care about what state the thread is in while terminating. But it can do bad things like leave partially written files laying around.

Another solution is to figure out some way for the main thread to interrupt the thread. Suppose the threads waits of socket traffic. You could close the socket and the thread would be woken by that event.

Another solution is to only run threads for short-lived tasks that you want to complete. Your ctrl-c gets delayed a bit but you eventually exit. You could even set them up to run off of a queue and send a special "kill" message to them when done. In fact, python thread pools are a good way to go.

Another solution is to have the thread check a Event to see if its time to exit.

tdelaney
  • 73,364
  • 6
  • 83
  • 116