python multithreading issue in cronjob

Question

I have a python program that uses the ThreadPool for multithreading. The program is one step in a shell script. When I execute the shell script manually on the command line, the entire flow works as expected. However, when I execute the shell script as a cronjob, it appears that the flow goes to the next steps before the python multithreading steps are completely finished.

Inside the python program, I do call AsyncResult.get(timeout) to wait for all the results to come back before moving on.

Can you post your cronjob? Also, what do you mean by "the program is one step?" — hymloth, Jul 20 '11 at 18:43
Have a look at this: http://stackoverflow.com/questions/5455425/django-python-how-should-this-cronjob-be-executed/5456321#5456321 — hymloth, Jul 22 '11 at 15:04

score 0 · Answer 1 · edited Oct 24 '19 at 20:33

0

Try setting "TERM=xterm" (or whatever env variable you have, figure out by command 'env' on your terminal) in your crontab.

edited Oct 24 '19 at 20:33

Corentin Pane

4,794
1
12
29

answered Oct 24 '19 at 19:38

vp9031

71
2

score 0 · Answer 2 · answered Mar 13 '12 at 22:58

Run your program via batch(1) (see the output of the command man batch) as well. If that works OK, but the cron version does not, then it is almost certainly a problem with your environment variable setup. To verify that, run printenv from your interactive shell to inspect your environment there. Then do the same thing inside the crontab (you will just need to temporarily set up an extra cron entry for it). Try setting the variables in your shell script before invoking Python.

On the other hand, if it doesn't work via batch(1) either, it could be something to do with the files that your code has open. Try running your shell script with input redirected from /dev/null and output going to a file:

$ /usr/local/bin/myscript </dev/null >|/tmp/outfile.txt 2>&1

python multithreading issue in cronjob

2 Answers2