Syntactically simpler flock invocation

Question

My use for flock is because we have N simultaneous processes starting that all need read access to a file. If the file doesn't exist, we need to create it, but only one process can do that, or else they would be writing all over each other.

The normal example of how to do that using Linux's flock is like this:

(
    flock -n 9 || exit 1
    if [ ! -f file.txt ]; then
        echo 'Simulate the file creation' > file.txt
    fi
) 9>/var/lock/mylockfile

However, this is very confusing to read, especially if you're not familiar with subshells and file descriptors. I'm wondering if it's possible to manually lock and unlock the file:

flock --exclusive file.txt

if [ ! -f file.txt ]; then
    echo 'Simulate the file creation' > file.txt
fi

flock --unlock file.txt

If not, is there some similar way to use flock, that's as readable as possible, avoiding subshells, exec, etc?

FYI https://blog.famzah.net/2013/07/31/using-flock-in-bash-without-invoking-a-subshell/ — marekful, Jul 09 '19 at 15:24
If you are going to do shell work, it is in everyone's best interest for you to *become* familiar with sub shells and file descriptors. — chepner, Jul 09 '19 at 16:11
https://stackoverflow.com/a/1985512/6770384 defines some functions that allow you to (un)lock manually using `flock`. However, using a tool like `lockfile`, `dotlockfile`, or `dotlock` seems more appropriate. There's also `sem` from GNU `parallel` which doesn't allow manual locking (afaik) but is very readable. — Socowi, Jul 09 '19 at 16:19
`lockfile` etc seem to have a much cleaner syntax. If you post an answer for this use-case with one of them I'd be happy to accept it. — Migwell, Jul 10 '19 at 02:29

score 0 · Answer 1 · answered Jul 09 '19 at 17:45

I suggest that you follow the idiom presented in the man page, and provide the reader the URL to the man page as a comment. In a sense, I'm suggesting this change to your code:

# flock usage:  Please see https://stackoverflow.com/questions/56955601/syntactically-simpler-flock-invocation
# Or see flock third form usage here:  http://man7.org/linux/man-pages/man1/flock.1.html

(
    flock -n 9 || exit 1
    if [ ! -f file.txt ]; then
        echo 'Simulate the file creation' > file.txt
    fi
) 9>/var/lock/mylockfile

That said, I think your logic is fine, but I would opt to use another lock file other than the file you are creating.

So here's how I would try your approach:

flock --exclusive /var/log/mylockfile

if [ ! -f file.txt ]; then
    echo 'Simulate the file creation' > file.txt
fi

flock --unlock /var/log/mylockfile

Full disclosure: I haven't tried this myself, nor do I use flock. I do agree that your approach is more intuitive and easier to read than the third usage idiom presented by the manpage. You may consider just doing the file creation before you kick off your parallel processes (but I'm sure that has already been considered and dropped due to your particular case).

Does it matter which file is locked here? If I use the actual file of interest (`file.txt`) as the lock file, will `flock` truncate it? — Migwell, Jul 10 '19 at 02:18
Also, I'm concerned that this example won't actually work, because I believe the first form of `flock` will open the lockfile then immediately close it because we have provided no command. And then it won't help as a lock. And also the `--unlock` then won't do anything. — Migwell, Jul 10 '19 at 02:22

Syntactically simpler flock invocation

1 Answers1