Using tail in a subshell in conjunction with while/break does not exit the loop

Question

I have been facing a very peculiar issue with shell scripts.

Here is the scenario

Script1 (spawns in background)--> Script2

Script2 has the following code

function check_log()
{
    logfile=$1
    tail -5f ${logfile} | while read line
    do
      echo $line
      if echo $line|grep "${triggerword}";then
        echo "Logout completion detected"
        start_leaks_detection
        triggerwordfound=true
        echo "Leaks detection complete"
      fi
      if $triggerwordfound;then
        echo "Trigger word found and processing complete.Exiting"
        break
      fi

    done
        echo "Outside loop"
        exit 0

}

check_log "/tmp/somefile.log" "Logout detected"

Now the break in while loop does not help here. I can see "Logout completion detected" as well as "Leaks detection complete" being echoed on the stdout, but not the string "outside loop"

I am assuming this has to do something with tail -f creating a subshell. What I want to do is, exit that subshell as well as exit Script2 to get control back to Script1.

Can someone please shed some light on how to do this?

Answer 1

Instead of piping into your while loop, use this format instead:

while read line
do
   # put loop body here
done < <(tail -5f ${logfile})

Answer 2

Try this, although it's not quite the same (it doesn't skip the beginning of the log file at startup):

triggerwordfound=
while [ -z "$triggerwordfound" ]; do
    while read line; do
        echo $line
        if echo $line|grep "${triggerword}";then
            echo "Logout completion detected"
            start_leaks_detection
            triggerwordfound=true
            echo "Leaks detection complete"
        fi
    done
done < "$logfile"
echo "Outside loop"

The double loop effectively does the same thing as tail -f.

Answer 3

Your function works in a sense, but you won't notice that it does so until another line is written to the file after the trigger word has been found. That's because tail -5 -f can usually write all of the last five lines of the file to the pipe in one write() call and continue to write new lines all in one call, so it won't be sent a SIGPIPE signal until it tries to write to the pipe after the while loop has exited.

So, if your file grows regularly then there shouldn't be a problem, but if it's more common for your file to stop growing just after the trigger word is written to it, then your watcher script will also hang until any new output is written to the file.

I.e. SIGPIPE is not sent immediately when a pipe is closed, even if there's un-read data buffered in it, but only when a subsequent write() on the pipe is attempted.

This can be demonstrated very simply. This command will not exit (provided the tail of the file is less than a pipe-sized buffer) until you either interrupt it manually, or you write one more byte to the file:

tail -f some_large_file | read one

However if you force tail to make multiple writes to the pipe and make sure the reader exits before the final write, then everything will work as expected:

tail -c 1000000 some_large_file | read one

Unfortunately it's not always easy to discover the size of a pipe buffer on a given system, nor is it always possible to only start reading the file when there's already more than a pipe buffer's worth of data in the file, and the trigger word is already in the file and at least a pipe buffer's size bytes from the end of the file.

Unfortunately tail -F (which is what you should probably use instead of -f) doesn't also try writing zero bytes every 5 seconds, or else that would maybe solve your problem in a more efficient manner.

Also, if you're going to stick with using tail, then -1 is probably sufficient, at least for detecting any future event.

BTW, here's a slightly improved implementation, still using tail since I think that's probably your best option (you could always add a periodic marker line to the log with cron or similar (most syslogd implementations have a built-in mark feature too) to guarantee that your function will return within the period of the marker):

check_log ()
{
        tail -1 -F "$1" | while read line; do
                case "$line" in
                *"${2:-SOMETHING_IMPOSSIBLE_THAT_CANNOT_MATCH}"*)
                        echo "Found trigger word"
                        break
                        ;;
                esac
        done
}

Replace the echo statement with whatever processing you need to do when the trigger phrase is read.