Java IO implementation of unix/linux "tail -f"

Question

I'm wondering what techniques and/or library to use to implement the functionality of the linux command "tail -f ". I'm essentially looking for a drop in add-on/replacement for java.io.FileReader. Client code could look something like this:

TailFileReader lft = new TailFileReader("application.log");
BufferedReader br = new BufferedReader(lft);
String line;
try {
  while (true) {
    line= br.readLine();
    // do something interesting with line
  }
} catch (IOException e) {
  // barf
}

The missing piece is a reasonable implementation of TailFileReader. It should be able to read parts of the file that exist before the file is opened as well as the lines that are added.

Answer 1

Take a look at Apache Commons implementation of Tailer class. It does seem to handle log rotation as well.

Answer 2

The ability to continue to read a file, and wait around until the file has some more updates for you shouldn't be that hard to accomplish in code yourself. Here's some pseudo-code:

BufferedReader br = new BufferedReader(...);
String line;
while (keepReading) {
    line = reader.readLine();
    if (line == null) {
        //wait until there is more of the file for us to read
        Thread.sleep(1000);
    }
    else {
        //do something interesting with the line
    }
}

I would assume that you would want to put this type of functionality in its own Thread, so that you can sleep it and not affect any other areas of your application. You would want to expose keepReading in a setter so that your main class / other parts of the application can safely shut the thread down without any other headaches, simply by calling stopReading() or something similar.

Answer 3

Check JLogTailer, which does this logic.

The main point in the code is:

public void run() {
    try {
        while (_running) {
            Thread.sleep(_updateInterval);
            long len = _file.length();
            if (len < _filePointer) {
                // Log must have been jibbled or deleted.
                this.appendMessage("Log file was reset. Restarting logging from start of file.");
                _filePointer = len;
            }
            else if (len > _filePointer) {
                // File must have had something added to it!
                RandomAccessFile raf = new RandomAccessFile(_file, "r");
                raf.seek(_filePointer);
                String line = null;
                while ((line = raf.readLine()) != null) {
                    this.appendLine(line);
                }
                _filePointer = raf.getFilePointer();
                raf.close();
            }
        }
    }
    catch (Exception e) {
        this.appendMessage("Fatal error reading log file, log tailing has stopped.");
    }
    // dispose();
}

Answer 4

I've built a short implementation of "tail -f" in Scala some time ago: tailf. It takes care of file rotation as well and you may define your own logic what to do when it reaches EOF or finds the file has been renamed.

You may take a look and port it to Java, since actually there is nothing complex in there. Few notes: the main file is Tail.scala and basically it defines FollowingInputStream which takes care of EOF/rename and follow method, which wraps FollowingInputStream into an unbounded enumeration in SequenceInputStream. So, as soon as FollowingInputStream ends, SequenceInputStream requests next element from an Enumeration and another FollowingInputStream gets created.

Answer 5

I stumbled recently over rxjava-file, It is an extension of RxJava. In contrast to the other solutions this makes use of Java's NIO.

import rx.Observable;
import rx.functions.Action1;
import com.github.davidmoten.rx.FileObservable;

// ... class definition omitted

public void tailLogFile() throws InterruptedException {
    Observable<String> tailer = FileObservable.tailer()
                                .file("application.log") // absolute path
                                .tailText();

    tailer.subscribe(
        new Action1<String>() {
            @Override
            public void call(String line) {
                System.out.println("you got line: " + line);
            }
        },
        new Action1<Throwable>() {
            @Override
            public void call(Throwable e) {
                System.out.println("you got error: " + e);
                e.printStackTrace();
            }
        }
    );

// this solution operates threaded, so something  
// is required that prevents premature termination

    Thread.sleep(120000);
}

Answer 6

Just was faced with the same issue - found the "simplest" implementation here: Java Tail.

*Great stuff * - ready for production ;)

I hope the code-citation will not drop some license.

    import java.io.BufferedReader;
    import java.io.FileReader;
    import java.io.IOException;

    /**
     * Java implementation of the Unix tail command
     * 
     * @param args[0] File name
     * @param args[1] Update time (seconds). Optional. Default value is 1 second
     * 
     * @author Luigi Viggiano (original author) http://it.newinstance.it/2005/11/19/listening-changes-on-a-text-file-unix-tail-implementation-with-java/
     * @author Alessandro Melandri (modified by)
     * */
    public class Tail {

      static long sleepTime = 1000;

      public static void main(String[] args) throws IOException {

        if (args.length > 0){

          if (args.length > 1)
        sleepTime = Long.parseLong(args[1]) * 1000;

          BufferedReader input = new BufferedReader(new FileReader(args[0]));
          String currentLine = null;

          while (true) {

        if ((currentLine = input.readLine()) != null) {
          System.out.println(currentLine);
          continue;
        }

        try {
          Thread.sleep(sleepTime);
        } catch (InterruptedException e) {
          Thread.currentThread().interrupt();
          break;
        }

          }
          input.close();

        } else {
          System.out.println("Missing parameter!\nUsage: java JavaTail fileName [updateTime (Seconds. default to 1 second)]");
        }
      }
    }

Answer 7

I found this nice tail implementation.

Author : amelandri

Souce from : https://gist.github.com/amelandri/1376896

import java.io.BufferedReader;
import java.io.FileReader;
import java.io.IOException;

/**
 * Java implementation of the Unix tail command
 * 
 * @param args[0] File name
 * @param args[1] Update time (seconds). Optional. Default value is 1 second
 * 
 * @author Luigi Viggiano (original author) http://it.newinstance.it/2005/11/19/listening-changes-on-a-text-file-unix-tail-implementation-with-java/
 * @author Alessandro Melandri (modified by)
 * */
public class Tail {

  static long sleepTime = 1000;

  public static void main(String[] args) throws IOException {

    if (args.length > 0){

      if (args.length > 1)
        sleepTime = Long.parseLong(args[1]) * 1000;

      BufferedReader input = new BufferedReader(new FileReader(args[0]));
      String currentLine = null;

      while (true) {

        if ((currentLine = input.readLine()) != null) {
          System.out.println(currentLine);
          continue;
        }

        try {
          Thread.sleep(sleepTime);
        } catch (InterruptedException e) {
          Thread.currentThread().interrupt();
          break;
        }

      }
      input.close();

    } else {
      System.out.println("Missing parameter!\nUsage: java JavaTail fileName [updateTime (Seconds. default to 1 second)]");
        }
      }

}

Answer 8

If your code only ever will have to run on Unix systems, you may be able to get away with just shelling out and calling tail -f directly.

As a more involved alternative, you could take a look at the implementation of GNU tail and port that over to Java. (I'm not sure whether this wouldn't already make your code a derivative work, though.)

Answer 9

Here's a short story which you could use as a pointer:

I've coded TailingInputStream at work for the very same reason. It basically uses File and refreshed its contents on demand and checked against internal buffer if it has changed significantly (4kB memory stamp IIRC) and then did what the tail -f does. A bit hacky, yes, but it works perfectly and doesn't mess with Threads or anything fancy like that - it's compatible all the way back to 1.4.2 at least.

That said, it was a lot easier to do than ReverseInputStream which went from file's end to start and didn't die if the file was updated on the fly...