Tailing Rolling Files

Question

I have a directory full of rolling log files that I would like to be able to use tail on.

The files are named as such:

name      modified
00A.txt   Dec 27 19:00
00B.txt   Dec 27 19:01
00C.txt   Dec 27 19:02
00D.txt   Dec 27 19:03

On an older unix system, I'm trying to come up with a shell script that will tail the most recently modified file in a specific directory, and if that file gets administratively closed (rolls to the next file) I want the program to automatically begin tailing the new file without me having to break out of tail to rerun.

tail -100f `ls -t | head -1`

The desired behavior, given the filenames above, would go like this:

./logtailer.sh

Then the script would begin tailing 00D.txt. As soon as the logger was finished writing to 00D.txt and the newest log file was now named 00E.txt, the program would automatically begin tailing that file.

One could write this script by watching the output of tail for the text "File Administratively Closed" and then having the following command run again.

tail -100f `ls -t | head -1`

Is there a more elegant way to do this than by watching for the text "file administratively closed"? How can I even read the output of tail line by line in a shell script?

Edit: I should explain that the -F flag for tail is not an option for me on this system. It uses a different version of tail that does not contain this feature. OS version - Solaris 10

score 20 · Answer 1 · answered Dec 27 '11 at 18:28

20

You can use the -F option for tail which implies --follow=name --retry.

From the man page:

-F      
The -F option implies the -f option, but tail will also check to see if the 
file being followed has been renamed or rotated.  The file is closed and 
reopened when tail detects that the filename being read from has a new inode 
number. The -F option is ignored if reading from standard input rather than 
a file.

answered Dec 27 '11 at 18:28

jaypal singh

74,723
23
102
147

I wish I could use this flag, but my version of tail does not have this feature and I do not have the option to upgrade that. (No administrative rights) – user1118047 Dec 27 '11 at 18:43
Can you please provide the `version info` of `tail` and the `box (unix/linux)` you are using? – jaypal singh Dec 27 '11 at 18:46
You will have to get in touch with your admin to install a third party utility called [Multitail](http://www.vanheusden.com/multitail/). – jaypal singh Dec 27 '11 at 19:07
2

-1, this is not what the OP is asking for. `tail --follow=name` is for when the input file is truncated by a log rotation script, but OP wants to start reading a new file when it springs into existemce. – tripleee Dec 27 '11 at 19:42
I suggested `tail -F` option which does exactly that. OP doesn't have that option on his Solaris box. – jaypal singh Dec 27 '11 at 19:45
multitail would certainly be nice but I've been told that new software installs are absolutely prohibited on these boxes. – user1118047 Dec 27 '11 at 19:49
I'm going to give this +1 because it would work if the new file created followed the standard rotation convention of .1, .2, etc. I don't believe it works with 00A.txt, 00B.txt but this is still the best answer. – SiegeX Dec 27 '11 at 21:10
I'm giving this a +1. It may not be exaclty what the op was looking for, but it is exactly what I was... – Adam Lerman Apr 10 '13 at 16:59

score 3 · Answer 2 · answered Dec 27 '11 at 20:40

you may need inotify to detect the creation of the new file, a workaround for that would be to keep polling the filesystem while running tail on the background:

#!/bin/bash

get_latest() {
    local files=(*.log)
    local latest=${files[${#files[@]}-1]}
    echo "$latest"
    [[ $latest == $1 ]]
}
run_tail() {
    tail -c +0 -f "$1"
}

while true; do
    while current=$(get_latest "$current"); do
        sleep 1
    done
    [[ $pid ]] && kill $pid
    run_tail "$current" & pid=$!
done

(untested, unnecesarily hacky and be careful with the limitations of your old system!)

I like where this answer is going...I wasn't able to plug-and-play this right into a shell script and have it do what I want, but I'm going to play around with the code here and see what I can do to make it work. — user1118047, Dec 27 '11 at 21:37

Tailing Rolling Files

2 Answers2

Linked