19

A typical approach to avoid two instances of the same script running simultaneously looks like this:

[ -f ".lock" ] && exit 1
touch .lock
# do something
rm .lock

Is there a better way to lock on files from a shell-script, avoiding a race condition? Must directories be used instead?

Chris W.
  • 1,680
  • 16
  • 35
n-alexander
  • 14,663
  • 12
  • 42
  • 43

5 Answers5

26

Yes, there is indeed a race condition in the sample script. You can use bash's noclobber option in order to get a failure in case of a race, when a different script sneaks in between the -f test and the touch.

The following is a sample code-snippet (inspired by this article) that illustrates the mechanism:

if (set -o noclobber; echo "$$" > "$lockfile") 2> /dev/null; 
then
   # This will cause the lock-file to be deleted in case of a
   # premature exit.
   trap 'rm -f "$lockfile"; exit $?' INT TERM EXIT

   # Critical Section: Here you'd place the code/commands you want
   # to be protected (i.e., not run in multiple processes at once).

   rm -f "$lockfile"
   trap - INT TERM EXIT
else
   echo "Failed to acquire lock-file: $lockfile." 
   echo "Held by process $(cat $lockfile)."
fi
Chris W.
  • 1,680
  • 16
  • 35
Barry Kelly
  • 41,404
  • 5
  • 117
  • 189
  • In case anyone else was wondering what the `$$` does, it gives back the PID (process ID) of the running script. – Chris W. Jul 15 '19 at 20:05
10

Try flock command:

exec 200>"$LOCK_FILE"
flock -e -n 200 || exit 1

It will exit if the lock file is locked. It is atomic and it will work over recent version of NFS.

I did a test. I have created a counter file with 0 in it and executed the following in a loop on two servers simultaneously 500 times:

#!/bin/bash

exec 200>/nfs/mount/testlock
flock -e 200

NO=`cat /nfs/mount/counter`
echo "$NO"
let NO=NO+1
echo "$NO" > /nfs/mount/counter

One node was fighting with the other for the lock. When both runs finished the file content was 1000. I have tried multiple times and it always works!

Note: NFS client is RHEL 5.2 and server used is NetApp.

jpastuszek
  • 799
  • 5
  • 8
  • Reasonably good coverage on flock: in my sampling, I have it on Cygwin and Linux, but not Solaris or Mac. – Barry Kelly Feb 04 '11 at 11:30
  • 2
    Would you mind explaining the syntax a bit more? I am especially wondering about `exec 200>"$LOCK_FILE"`. Gonna figure this out with the man pages, but your answer would be much better if it explained what these lines do. – Dr. Jan-Philip Gehrcke Aug 01 '13 at 13:42
  • Bash 3.2 manual, section 3.6: "Redirections using file descriptors greater than 9 should be used with care, as they may conflict with file descriptors the shell uses internally." Intense use of FD 200 (such as 500 times in a row from multiple processes) could cause problems, no? – system PAUSE Jan 04 '14 at 01:04
4

Lock your script (against parallel run)

http://wiki.bash-hackers.org/howto/mutex

FYI.

Hank
  • 143
  • 6
0

Creating a directory is atomic, and fails if it already exists if you don't use -p, so

 mkdir $lockName || exit 1

works.

...but use flock instead.

Paul Hodges
  • 13,382
  • 1
  • 17
  • 36
-1

seems like I've found an easier solution: man lockfile

n-alexander
  • 14,663
  • 12
  • 42
  • 43
  • 1
    Note that lockfile is not portable - it might not available; it's part of procmail (AFAIK). – Barry Kelly Nov 28 '08 at 12:34
  • The shell is more portable than procmail, in so far as you're more likely to have bash than procmail. I have bash on Solaris, Linux, Mac and Windows here. `lockfile` isn't on any of them. – Barry Kelly Feb 04 '11 at 11:25
  • 1
    I wrote a bash script using noclobber that mimics some of the behavior of Procmail's lockfile. It should be portable, and you can find it here: http://www.codng.com/2011/05/file-locks-in-bash.html – juancn May 18 '11 at 14:53