Watch file and execute command using tail -f and while read?

Question

I am trying to watch a file and execute a command once every time the file is changed, ideally with just native bash commands.

This is as far as I have got, but how do I check if I have reached the beginning or end of the file? I realize that tail -f doesn't read EOF so how I can tell that I have reached the end of the file?

 tail -f source_file.js | while read line || [[ -n "$line" ]]; 
     # how do I execute a command here just **once**?
 done

Answers which don't use tail or while read will be accepted as long as they are native bash commands and about one line.

Perhaps I could zero a variable every time while is called?

Answer 1

From the man:

-f

The -f option causes tail to not stop when end of file is reached, but rather to wait for additional data to be appended to the input.

So, if you want monitor and do actions, need break the script into two processes

• one will show the content tail (if you want personally monitor the changes)
• second will monitor changes and do actions

or

• you should use for example perl or python what can monitor end-of-file and execute some actions when reach it (for example, run an bash script).

The bash soultion can be based of file-modification time

file="./file"

runcmd() {
        echo "======== file $1 is changed ============"
}

#tail -f "$file" &   #uncomment 3 lines is you want pesonally watch the file
#tailpid=$!
#trap "kill $tailpid;exit" 0 2    #kill the tail, when CTRL-C this script

lastmtime=0
while read -r mtime < <(stat -c '%Z' "$file")
do
        if [[ $lastmtime != $mtime ]]
        then
                lastmtime=$mtime
                runcmd "$file"
        fi
        sleep 1
done

added another solution based on standard perl

perltail() {
#adapted from the perlfaq5
#http://perldoc.perl.org/perlfaq5.html#How-do-I-do-a-tail--f-in-perl%3f
perl -MTime::HiRes=usleep -Mstrict -Mautodie -e '
$|=1;
open my $fh, "<", "$ARGV[0]";
my $curpos;
my $eof=1;
for (;;) {
    for( $curpos = tell($fh); <$fh>; $curpos =tell($fh) ) {
        print;
        $eof=1
    }
    print "=EOF-reached=\n" if $eof;
    $eof=0;
    usleep(1000); #adjust the microseconds
    seek($fh, $curpos, 0);
}' "$1"
}

eof_action() {
    echo "EOF ACTION"
    printf "%s\n" "${newlines[@]}"  #new lines received from the last eof
    newlines=()         #empty the newlines array
}

perltail "./file" | while read -r line
do
    if [[ $line =~ =EOF-reached= ]]
    then
        eof_action
        continue
    fi
    #do something with the received lines - if need
    #for example, store new lines into variable for processing in the do_action and like
    newlines+=($line)
done

Principe:

• the perltail bash function runs an perl implementation of tail -f, and additionally to, when reached the end-of-file it prints an MARK to the output, here: =EOF-reached=.
• the bash while read looking for the MARK and run action only the the mark exists - e.g. only when the end-of-file reached.

Answer 2

It's by no means a native bash solution but you could use libinotify to do what you want:

while inotifywait -qqe modify file; do 
    echo "file modified"
done

This watches for modifications to file and performs the action within the loop whenever they happen. The -qq switch suppresses the output of the program, which by default prints a message every time something happens to the file.

Answer 3

I am not sure of your exact meaning of doing something "once every time a file is changed" or an implementation of "about one line". The tail command is usually used in a line-oriented manner, so I guess that something like 500 lines of text appended to a file in a single write(2) is an update worthy of only one invocation of you command.

But what about, say tens of lines appended after delays of tens of milliseconds? How often do you wish the command be called?

If libinotify is available, per Mr. Fenech, use it. If you're trying to use somewhat more basic shell utilities, find(1) can also be used to notice when one file has become newer than another.

For example, a script that watches a file, and runs a supplied command when it's updated, polling at 1-second intervals.

#!/bin/sh

[ $# -ge 2 ] || { echo "Usage: $(basename $0) <file> <cmd...>" 1>&2; exit 1; }
[ -r "$1"  ] || { echo "$1: cannot read" 1>&2; exit 1; }

FILE=$1; shift

FTMP=$(mktemp /tmp/$(basename "$FILE")_tsref.XXXXXX)
trap 'rm -f "$FTMP"' EXIT

touch -r "$FILE" "$FTMP"

while true; do
    FOUT=$(find "$FILE" -newer "$FTMP") || exit $?
    if [ "$FOUT" = "$FILE" ]; then
        touch -r "$FILE" "$FTMP"
        eval "$@"
    else
        sleep 1
    fi
done

Answer 4

Lot of already written to this question and probably need clarify some principes.

If you want wait to EOF while the monitor process reading lines (regardless what is it, e.g. tail or anything other) you must specify the wait-interval, with other words, what is considered as "new lines" in the file.

Imagine:

• you will reach EOF, but in 100 microseconds new line arrived, the process will read it and reach another EOF. Belong this new line to the previous block of lines? (probably yes).
• And what if the new line arrive in 5 seconds? This is probably a new block of lines.

So, as you can see, the specifying the time what you considering for the "new block" of lines is abosolutely necessary. With other words, if you reach EOF multiple times in an specified time interval - it mean one EOF only. (like reach EOF twice in 100 microseconds).

Therefore the tail -f itself using 1 seconds timeout as default, and in the GNU version you can change this with the -s parameter. From the docs:

-s, --sleep-interval=N

with -f, sleep for approximately N seconds (default 1.0) between iterations; with inotify and --pid=P, check process P at least once every N seconds

Also, you can check this in the source code of the tail.

For the the inotify library and inotofy-tools the principe is the same. (And tail, (depends on of your distribution) can use inotofylib itself)). For using the inotifylib must call the function

struct inotify_event* inotifytools_next_event (int timeout)

Your program should call this function or inotifytools_next_events() frequently; between calls to this function, inotify events will be queued in the kernel, and eventually the queue will overflow and you will miss some events. (see here).

Again, the time-interval is essential (and all common tools defaults it to 1 sec).

About the solution what you tried with -n $line. It can't work, because the tail never returns an empty line, when reaching EOF. The tail simply return the last line what got, and waits for the new lines (and checks them in specified time-intervals).

Summary:

• you must specify the timeout on what you want check the EOF condition (probably 1 second should be OK, if not - change the sleep time of the above scripts.
• all above solutions are working and are OK
• and finally:

Answers which don't use tail or while read will be accepted as long as they are native bash commands and about one line.

isn't make any sense, because of the above.

Hope this helps.

Answer 5

Provided the editor used is not using atomic save the this works

tail -f source_file.js  2>&1 > /dev/null | while read line; do echo 'do any command here'; done 

(Atomic save can be turned off easily in sublime )

Answer 6

This will allow you to use tail:

while read line; do
#code goes here
done < <(tail -f $FILE)

However, the loop runs once for every line (not once per file change). The benefit over using < <(stat -c '%Z' "$file") with sleep is that it catches every change. To do that with any reasonable certainty using stat, you have to drop the sleep command, but that hogs CPU resources.

You could probably use a global variable with additional logic in the loop to stop the main command from running again within "x lines and/or y seconds" to prevent the command from being run for each individual line, but that's still unreliable and quickly becomes cumbersome.

Your best bet really is to use inotifywait (from inotify-tools), since it sets up a listener for the file, instead of polling every second (or continuously) like you'd do with stat.

[EDIT]

Alternatively, if there's some key word or sequence that's added to the file exactly once every time it's updated, just check for it with if $(grep --quiet "keyword" <<< $line) and put your code inside the if statement.