crontab to run python file if not running already

Question

I want to execute my python file via crontab only if its down or not running already. I tried adding below entry in cron tab but it does not work

24 07 * * * pgrep -f test.py || nohup python /home/dp/script/test.py & > /var/tmp/test.out

test.py works fine if i run pgrep -f test.py || nohup python /home/dp/script/test.py & > /var/tmp/test.out manually and it also works in crontab if i remove pgrep -f test.py || from my crontab and just keep 24 07 * * * nohup python /home/dp/script/test.py & > /var/tmp/test.out

Any idea why crontab does not work if i add pgrep -f? is there any other way i can run test.py just one time to avoid multiple running processes of test.py? Thanks, Deepak

Possible duplicate of [How do I write a bash script to restart a process if it dies?](http://stackoverflow.com/questions/696839/how-do-i-write-a-bash-script-to-restart-a-process-if-it-dies) — tripleee, Sep 19 '16 at 11:19

score 7 · Accepted Answer · edited Jun 20 '20 at 09:12

7

pgrep -f lists itself as a false match when run from cron

I did the test with a script.py running an infinite loop. Then

pgrep -f script.py

...from the terminal, gave one pid, 13132 , while running from cron:

pgrep -f script.py > /path/to/out.txt

outputs two pids, 13132 and 13635.

We can therefore conclude that the command pgrep -f script.py lists itself as a match, when run from cron. Not sure how and why, but most likely, this is indirectly caused by the fact that cron runs with a quite limited set of environment variables (HOME, LOGNAME, and SHELL).

The solution

Running pgrep -f from a (wrapper) script makes the command not list itself, even when run from cron. Subsequently, run the wrapper from cron:

#!/bin/bash

if ! pgrep -f 'test.py'
then
    nohup python /home/dp/script/test.py & > /var/tmp/test.out
# run the test, remove the two lines below afterwards
else
    echo "running" > ~/out_test.txt
fi

edited Jun 20 '20 at 09:12

Community

1
1

answered Sep 19 '16 at 11:01

Jacob Vlijm

3,099
1
21
37

`if [ -z $(command) ]` is usually an antipattern for `if command`, provided `command` is correctly written. – tripleee Sep 19 '16 at 11:20
@tripleee not sure what you mean, `if [ -z "$(pgrep -f 'test.py')" ]` is a valid test. – Jacob Vlijm Sep 19 '16 at 11:26
It is not incorrect, but it is highly dubious stylistically. The reason commands produce an exit code is so that you can use them in `if` or `while` condition. – tripleee Sep 19 '16 at 14:04
@triplee thanks for mentioning, that is new to me but makes sense. Fixed now. – Jacob Vlijm Sep 19 '16 at 15:12
The second `pgrep` inside the `if` seems superfluous, too. – tripleee Sep 19 '16 at 17:44
@tripleee Ah, you're right of course, blindly copied OP's command, fixed.Thanks once again. – Jacob Vlijm Sep 19 '16 at 18:16
Thanks all for the great comments and helping me with the options. – Deepak Prasad Sep 20 '16 at 06:51
great solution !! – Kartikeya Mishra Jun 26 '18 at 19:48

score 3 · Answer 2 · edited Sep 19 '16 at 15:16

It is generally a good idea to check whether the application is running or not within the application rather than checking from outside and starting it. Managing the process within the process rather than expecting another process to do it.

Let the cron run the application always
At the start of execution of the application, have a lock file or similar mechanism that will tell whether the process is already running or not.
If the application is already running, update somewhere the last test time and break the new process.
If the application is not running, log somewhere and notify someone (if required) and trigger the process start.

This will ensure that you will have more control over the life time of a process AND let you decide what to do in case of failures.

Not to mention, if you want to make sure that the application up time is to be high, it is best to make use of monitoring services such as Monit. Again this will depend on your application to add a sanity layer that says whether it is OK or not.

Rolf of Saxony · Answer 3 · 2016-09-19T14:24:14.877

cron may be seeing the pgrep -f test.py process as well as the test.py process, giving you the wrong result from pgrep.
Try the command without the -f, this should just look for an occurrence of test.py or replace -f with -o which will look for the oldest occurrence.

Your other option is to insert into test.py something along the lines of:

Pid = os.popen('pgrep -f test.py').read(5).strip()

which will allow you to check within the code itself, if it is already running.

NST · Answer 4 · 2017-06-06T19:21:11.220

Came across this old question looking for solution myself.

Use psutil:

import psutil
import sys
from subprocess import Popen

for process in psutil.process_iter():
    if process.cmdline() == ['python', 'your_script.py']:
        sys.exit('Process found: exiting.')

print('Process not found: starting it.')
Popen(['python', 'your_script.py'])

You can also use start time of the previous process to determine if it's running too long and might be hung:

process.create_time()

There is tons of other useful metadata of the process.

sagittarian · Answer 5 · 2018-04-04T12:27:31.057

The problem is the pgreg -f is recognizing the shell script that cron uses to run the command itself.

I solved the pgrep -f problem by encoding the file name in rot13 and decoding it with a python one-liner, since I didn't want to add a script wrapper just for the cron job. i.e.:

pgrep -f $(python3 -c 'import codecs; print(codecs.decode("grfg.cl", "rot13"))') || nohup python /home/dp/script/test.py & > /var/tmp/test.out

Added: You can use tr to decode rot13 which would result in a shorter and easier to read command:

pgrep -f $(echo grfg.cl | tr a-zA-Z n-za-mN-ZA-M) || nohup python /home/dp/script/test.py & > /var/tmp/test.out

score 0 · Answer 6 · answered Oct 05 '18 at 10:57

0

Why just check pid is already running like this

Pid = os.popen('pgrep -f myscript.py').readlines()
if len(Pid) > 1:
 exit()

answered Oct 05 '18 at 10:57

Konstantin Streletsky

331
1
8

score 0 · Answer 7 · answered Mar 25 '19 at 15:30

You can create a test file by the name test.py If you try executing the same file again it should quit.

import time
import os
from tempfile import TemporaryFile
import subprocess


def cron_running(p_name):
    run_cnt=0
    with TemporaryFile() as tf:
        subprocess.call(["ps","ax"], stdout=tf)
        tf.seek(0)
        for line in tf:
            if p_name in line.decode('ascii'):
                run_cnt+=1
    if run_cnt > 1:
        quit()

if cron_running(os.path.basename(__file__)):
    print("cron is already running")
    quit()

for i in range(60):
    time.sleep(1)

crontab to run python file if not running already

7 Answers7

pgrep -f lists itself as a false match when run from cron

The solution