How to check content of file that is being updated externally using java?

Question

There is a shell-script that is updating a log file.

I want to display the log from the log file while it is being written, line by line (and not the whole file at once).

Writing in the file is working properly, but I am having problems reading the file, as it displays the whole file at once.

Here is my code for reading:

String nmapstatusfile = runMT.instdir+"/logs/nmapout."+customer+".log";
String nmapcommand=runMT.instdir+"/bin/doscan.sh "+nmapoutfile+" "
    +IPRange+" "+nmapstatusfile+" "+nmapdonefile;

System.out.println("nmapcommand is .. "+nmapcommand);
Runtime r = Runtime.getRuntime();
Process pr = r.exec(nmapcommand);
RandomAccessFile raf =null;
try {
    System.out.println(nmapstatusfile);
    System.out.println("Reading a file");
raf = new RandomAccessFile(nmapstatusfile, "r");            
} catch (FileNotFoundException e1) {
    // TODO Auto-generated catch block
    e1.printStackTrace();
} // read from input file
System.out.println("Will print file contents line by line...");
long pos = 0;
raf.seek(pos);
String line =null;
while ((line = raf.readLine())!= null) {
    System.out.println("in while loop for file read.");
    System.out.println(line);
}
pos = raf.getFilePointer();
pr.waitFor();   

I have used RandomAccessFile, but have been unable to get the desired result.

How should I read file contents so that I can display line by line?

Answer 1

This requires Java 7, because it registers a watcher on the filesystem that will be notified whenever a given file changes. It allows any number of readers (using synchronization to avoid race conditions).

import java.io.BufferedReader;
import java.io.File;
import java.io.FileReader;
import java.io.IOException;
import java.nio.file.ClosedWatchServiceException;
import java.nio.file.Path;
import java.nio.file.StandardWatchEventKinds;
import java.nio.file.WatchKey;
import java.nio.file.WatchService;
import java.util.ArrayDeque;
import java.util.logging.Level;
import java.util.logging.Logger;

/**
 * Reads lines from a file that is actively being updated by another process
 */
public class Tailer {
    private static final Logger logger = Logger.getLogger("Tailer");

    private long tooLateTime = -1;
    private final long maxMsToWait;
    private final File file;
    private long offset = 0;
    private int lineCount = 0;
    private boolean ended = false;
    private WatchService watchService = null; 
    ArrayDeque<String> lines = new ArrayDeque<>();

    /**
     * Allows output of a file that is being updated by another process.
     * @param file to watch and read
     * @param maxTimeToWaitInSeconds max timeout; after this long without changes,
     * watching will stop. If =0, watch will continue until <code>stop()</code>
     * is called.
     */
    public Tailer(File file, long maxTimeToWaitInSeconds) {
        this.file = file;
        this.maxMsToWait = maxTimeToWaitInSeconds * 1000;
    }

    /**
     * Start watch.
     */
    public void start() {
        updateOffset();
        // listens for FS events
        new Thread(new FileWatcher()).start();  
        if (maxMsToWait != 0) {
            // kills FS event listener after timeout
            new Thread(new WatchDog()).start();
        }     
    }

    /**
     * Stop watch.
     */
    public void stop() {
        if (watchService != null) {
            try {
                watchService.close();
            } catch (IOException ex) {
                logger.info("Error closing watch service");
            }
            watchService = null;
        }
    }

    private synchronized void updateOffset() {
        tooLateTime = System.currentTimeMillis() + maxMsToWait;
        try {
            BufferedReader br = new BufferedReader(new FileReader(file));
            br.skip(offset);            
            while (true) {
                String line = br.readLine();
                if (line != null) {
                    lines.push(line);
                    // this may need tweaking if >1 line terminator char
                    offset += line.length() + 1; 
                } else {
                    break;
                }
            }
            br.close();
        } catch (Exception ex) {
            logger.log(Level.SEVERE, "Error reading", ex);
        }        
    }

    /**
     * @return true if lines are available to read
     */
    public boolean linesAvailable() {
        return ! lines.isEmpty();
    }

    /**
     * @return next unread line
     */
    public synchronized String getLine() {
        if (lines.isEmpty()) {
            return null;
        } else {
            lineCount ++;
            return lines.removeLast();
        }
    }

    /**
     * @return true if no more lines will ever be available, 
     * because stop() has been called or the timeout has expired
     */
    public boolean hasEnded() {
        return ended;
    }

    /**
     * @return next line that will be returned; zero-based
     */
    public int getLineNumber() {
        return lineCount;
    }

    private class WatchDog implements Runnable {
        @Override
        public void run() {
            while (System.currentTimeMillis() < tooLateTime) {
                try {
                    Thread.sleep(1000);
                } catch (InterruptedException ex) {
                    // do nothing
                }
            }
            stop();
        }
    }

    private class FileWatcher implements Runnable {
        private final Path path = file.toPath().getParent();
        @Override
        public void run() {
            try {
                watchService = path.getFileSystem().newWatchService();
                path.register(watchService, StandardWatchEventKinds.ENTRY_MODIFY);
                while (true) {
                    WatchKey watchKey = watchService.take();
                    if ( ! watchKey.reset()) {
                        stop();
                        break;
                    } else if (! watchKey.pollEvents().isEmpty()) {
                        updateOffset();
                    }
                    Thread.sleep(500);
                }
            } catch (InterruptedException ex) {
                logger.info("Tail interrupted");
            } catch (IOException ex) {
                logger.log(Level.WARNING, "Tail failed", ex);
            } catch (ClosedWatchServiceException ex) {
                // no warning required - this was a call to stop()
            }
            ended = true;
        }
    }

    /**
     * Example main. Beware: the watch listens on a whole folder, not on a single
     * file. Any update on a file within the folder will trigger a read-update.
     * @param args
     * @throws Exception 
     */
    public static void main(String[] args) throws Exception {
        String fn = args.length == 0 ? "/tmp/test/log.txt" : args[0];
        Tailer t = new Tailer(new File(fn), 10);
        t.start();
        while ( ! t.hasEnded()) {
            while (t.linesAvailable()) {
                System.out.println(t.getLineNumber() + ": " + t.getLine());
            }
            Thread.sleep(500);
        }
    }
}

Answer 2

You could implement polling strategy for the reader, to avoid extra notifications from writer to the reader.
Below is working simplified example :

• Writer updates log file from main thread (by user input from console)
• Reader periodically polls log file and outputs a single line

---

public class Test {
    public static void main(String[] args) throws IOException {

        // async log reader based on timerTask
        LogReader logReader = new LogReader("test.txt");
        logReader.start(); // starts polling log file in Timer thread

        // main thread simulating writes to file from user input
        BufferedReader console = new BufferedReader(new InputStreamReader(System.in));
        BufferedWriter logWriter = new BufferedWriter(new FileWriter(new File("test.txt")));
        String line = null;
        while (!(line = console.readLine()).equals("end")) {
            logWriter.write(line);
            logWriter.newLine();
            logWriter.flush();
        }
        logReader.stop();
        logWriter.close();
    }

    /**
     * reads file from separate thread
     */
    static class LogReader {
        File mLogFile = null;
        BufferedReader mReader = null;
        TimerTask mCheckFileTask = null;

        LogReader(String logFile) throws FileNotFoundException {
            mLogFile = new File(logFile);
            mReader = new BufferedReader(new FileReader(mLogFile));
        }

        /**
         * launches checking thread and output thread
         */
        void start() {
            mCheckFileTask = new TimerTask() {
                @Override
                public void run() {
                    // possibly some extra checks :
                    // file size or lastModified() time changed since last time
                    try {
                        writeChange();
                    } catch (IOException e1) {
                        e1.printStackTrace();
                        throw new Error(e1);
                    }
               }
            };

            new Timer().schedule(mCheckFileTask, 500, 500);
        }

        private void writeChange() throws IOException {
            String line;
            line = mReader.readLine();
            if (line != null) {
                System.out.println("line added: " + line);
            }
        }

        void stop() throws IOException {
            System.out.println("stopping reader");
            mCheckFileTask.cancel();
            mReader.close();
        }
    }
}

You might want add extra file-locking and consistency checks when having multiple readers/writers running in separate threads/processes.

Answer 3

public static void main(String[] args) throws IOException {
    FileWriter fw = new FileWriter(new File("C://test.txt"));
    BufferedWriter bw = new BufferedWriter(fw);
    bw.append("Hi");
    //bw.close();
    FileReader fr = new FileReader(new File("C://test.txt"));
    BufferedReader br = new BufferedReader(fr);
    System.out.println(br.readLine());
}

OP : null
if you uncomment bw.close() , then OP : Hi..

So, the problem is each time your script writes into your log file, you need to save the file, you need to notify() the waiting thread, and show it on the console...