Make inotifywait group multiple file updates into one?

Question

I have a folder with Sphinx docs that I watch with inotifywait (from inotify-tools). The script re-builds the html & singlehtml and refreshes Chrome.

#!/bin/sh
inotifywait -mr source --exclude _build -e close_write -e create -e delete -e move | while read file event; do
    make html singlehtml
    xdotool search --name Chromium key --window %@ F5
done

This works fine when I save a single file. However, when I hg update to an old revision or paste multiple files in source folder, it fires the script for every single file.

Is there a simple workaround (without writing custom python scripts -- this I can do) to make it wait a fraction of a second before firing the script?

See also: https://unix.stackexchange.com/q/390914/26227 – Dave Jarvis Jun 22 '19 at 22:53 — Dave Jarvis, Jun 22 '19 at 22:53

score 11 · Accepted Answer · edited Jun 22 '19 at 22:52

I made a bit more complex shell script and posted it in the article:

inotifywait -mr source --exclude _build -e close_write -e create -e delete -e move --format '%w %e %T' --timefmt '%H%M%S' | while read file event tm; do
    current=$(date +'%H%M%S')
    delta=`expr $current - $tm`
    if [ $delta -lt 2 -a $delta -gt -2 ] ; then
        sleep 1  # sleep 1 set to let file operations end
        make html singlehtml
        xdotool search --name Chromium key --window %@ F5
    fi
done

It makes inotifywait log not only filename & action, but also timestamp. The script compares the timestamp with current unixtime and if the delta is less than 2 sec, it runs make html. But before that it sleeps 1 second to let file operations end. For the next modified files the timestamp will be old, the delta will be more than 2 seconds, and nothing will be done.

I found this way was the least CPU consuming and the most reliable.

I also tried running a simple Python script, but this meant if I pasted something as big as jQueryUI into the folder, a thousand processes were spawned and then became zombies.

Owen T. Heisler · Answer 2 · 2018-08-31T17:19:42.253

Try this:

last_update=0
inotifywait -mr source --exclude _build -e close_write -e create \
    -e delete -e move --format '%T' --timefmt '%s' |
    while read timestamp; do
        if test $timestamp -ge $last_update; then
            sleep 1
            last_update=$(date +%s)
            make html singlehtml
            xdotool search --name Chromium key --window %@ F5
        fi
    done

--format '%T' --timefmt '%s' causes a timestamp to be output for each event.
test $timestamp -ge $last_update compares the event timestamp with the timestamp of the last update. Any events that occurred during the sleep are thus skipped.
sleep 1 is added to wait for events to accumulate. A shorter duration might be good here, like sleep 0.5, but it would be less portable.
last_update=$(date +%s%N) sets a timestamp for the last update to be compared with the next event's timestamp. In this way, any additional events that occur during the sleep 1 are discarded during the next iteration of the loop.

Note, there is a race condition here because strftime() does not support nanoseconds. This example may run make twice if a group of events crosses a second boundary. To instead risk missing events, replace -ge with -gt.

I think the `last_update` is not set for the subsequent events. I tried something similar and it did not work. I'd suggest better go with the accepted answer — Rakesh N, Aug 30 '18 at 09:00
The `last_update` variable is not intended to be set for each event, but rather for each execution of the "update" command. The `last_update` value is then compared with each subsequent event timestamp to determine whether another "update" command is necessary. I attempted to clarify this above. — Owen T. Heisler, Aug 31 '18 at 17:12
I have also moved the `sleep 1` into the `if` statement, where it should be, to prevent a sleep from occurring for every single event. — Owen T. Heisler, Aug 31 '18 at 17:23

Make inotifywait group multiple file updates into one?

2 Answers2