Dynamic monitoring of log file

Question

I need to monitor a log file for a pattern. The log file continually gets written by an application.

The application can add new log statements while my program is reading it.
The log gets rolled over when it’s >200 MB or at end of the day, so my program should handle change in filename dynamically.
If my program crashes for any reason, it has to resume from where it left off.

I do not want to re-invent the wheel. I am looking for a Java API. I wrote a program to read file and put in a loop with 30 seconds sleep, but that does not meet all the criteria.

>= Java 8: https://docs.oracle.com/javase/tutorial/essential/io/notification.html — tb-, Oct 28 '15 at 21:37

score 4 · Accepted Answer · answered Oct 03 '11 at 14:31

4

You might consider looking at apache commons io classes, in particular Tailer/TailerListener classes. See http://www.devdaily.com/java/jwarehouse/commons-io-2.0/src/main/java/org/apache/commons/io/input/Tailer.java.shtml.

answered Oct 03 '11 at 14:31

Kevin

24,871
19
102
158

score 2 · Answer 2 · answered Oct 03 '11 at 14:27

These two API's can be helpful:

1

JxFileWatcher (Official Site)

Read here what it is capable of

2

Jnotify

JNotify is a java library that allow java application to listen to file system events, such as:

File created
File modified
File renamed
File deleted

score 2 · Answer 3 · edited May 23 '17 at 12:24

2

If you are using Log4j, or can integrate it, it is possible to append log outputs to a convenient object, such as a StringBuffer, as it has been discussed in this related question: Custom logging to gather messages at runtime

edited May 23 '17 at 12:24

Community

1
1

answered Oct 03 '11 at 14:27

Xavi López

27,550
11
97
161

I did look at socketAppender from log4j, intially it caused my log producer to slow down significantly. Next was to wrap log producer in AsyncAppender. That did free up Producer but then intermittently I would see 'EOF socket closed'. my testcase would generate 1mil events in 5mins window for perf test. – nuvatech Oct 03 '11 at 17:54

score 1 · Answer 4 · edited May 23 '17 at 12:33

1

This looks similar: Implementation of Java Tail

Essentially you use a BufferedReader. Tracking where you left off will be something you'll have to add, perhaps capture the last line read?

That same question references JLogTailer which looks interesting and may do most of what you want already.

edited May 23 '17 at 12:33

Community

1
1

answered Oct 03 '11 at 14:27

Freiheit

8,408
6
59
101

Dynamic monitoring of log file

4 Answers4

1

2