Running script constantly in background: daemon, lock file with crontab, or simply loop?

Question

I have a Perl script that

• queries a database for a list of files to process
• processes the files
• and then exits

Upon startup this script creates a file (let's say script.lock), and upon exit it removes this file. I have a crontab entry that runs this script every minute. If the lockfile exists then the script exits, assuming that another instance of itself is running.

The above process works fine but I am not very happy with the robustness of this approach. Specifically, if for some reason the script exits prematurely and the lockfile is not removed then a new instance will not execute properly.

I would appreciate some advice on the following:

1. Is using the lock file a good approach or is there a better/more robust way to do this?
2. Is using crontab for this a good idea or could I better write an endless loop with sleep()?
3. Should I use the GNU 'daemon' program or the Perl Proc::Daemon module (or some other equivalent) for this?

Answer 1

Let's assume you take the continuous loop route. You rejigger your program to be one infinite loop. You sleep for a certain amount of time, then wake up and process your database files, and then go back to sleep.

You now need a mechanism to make sure your program is still up and running. This could be done via something like inetd.

However, your program basically does a single task, and does that task repeatedly through the day. This is what crontab is for. The inetd mechanism is for servers that are waiting for a client, like https or sshd. In these cases, you need a mechanism to recreate the server process as soon as it dies.

One way you can improve your lockfile mechanism is to include the PID with it. For example, in your Perl script you do this:

open my $lock_file_fh, ">", LOCK_FILE_NAME;
say {$lock_file_fh} "$$";
close $lock_file_fh;

Now, if your crontab sees the lock file, it can test to see if that process ID is still running or not:

if [ -f $lock_file ]
then
    pid=$(cat $lock_file)
    if ! ps -p $pid
    then
        rm $lock_file
    fi
    restart_program
else
    restart_program
fi

Answer 2

1. Using a lock file is a fine approach if using cron, although I would recommend a database if you can install and use one easily (MySQL/Postgres/whatever. NOT SQLite). This is more portable than a file on a local filesystem, among other reasons, and can be re-used.

2. You are indeed correct. cron is not the best idea for this scenario just for the reason you described - if the process dies prematurely, it's hard to recover (you can, by checking timestamps, but not very easily).

   What you should use cron for is a "start_if_daemon_died" job instead.

3. This is well covered on StackOverflow already, e.g. here:

   " How can I run a Perl script as a system daemon in linux? " or more posts.

Answer 3

This is not meant as a new answer but simply a worked out example in Perl of David W.'s accepted answer.

my $LOCKFILE = '/tmp/precache_advs.lock';

create_lockfile();
do_something_interesting();
remove_lockfile();

sub create_lockfile {
    check_lockfile();

    open my $fh, ">", $LOCKFILE or die "Unable to open $LOCKFILE: $!";
    print $fh "$$";
    close $fh;

    return;
}

sub check_lockfile {
    if ( -e $LOCKFILE ) {
        my $pid = `cat $LOCKFILE`;
        if ( system("ps -p $pid") == 0 ) {
            # script is still running, so don't start a new instance
            exit 0;
        }
        else {
            remove_lockfile();
        }
    }
    return;
}

sub remove_lockfile {
    unlink $LOCKFILE or "Unable to remove $LOCKFILE: $!";
    return;
}