Read data from log file periodically

Question

I am generating a log file and what i want is that i want to read the data periodically without having to read from the beginning each time. can anyone help.

an example of what you have done already would help too. ;) – Peter Lawrey Nov 30 '11 at 10:12 — Peter Lawrey, Nov 30 '11 at 10:12

score 4 · Accepted Answer · answered Nov 30 '11 at 10:11

4

Open the file and have a loop which,

get the size and compare with the size you have read already.
if the size has grown, read that many bytes and no more. Doing this means you can read more later.
if the size has shrink, close the file and start again.

You can use FileInputStream or RandomAccessFile.

answered Nov 30 '11 at 10:11

Peter Lawrey

525,659
79
751
1,130

If the reader is in the same process as the writer, you can place the length in a synchronized variable with notifyAll()/wait() and have an average latency of about 100 micro-seconds between writer and reader. – Peter Lawrey Nov 30 '11 at 11:57
There is example code of this pattern available at http://www.informit.com/guides/content.aspx?g=java&seqNum=226 – ridale Apr 15 '14 at 02:45

score 1 · Answer 2 · edited Jan 25 '12 at 10:51

1

If you want to Run a program to read your log file periodically then you can use schedulers like, Quartz Scheduler to run it periodically.

edited Jan 25 '12 at 10:51

mmmmmm

32,227
27
88
117

answered Nov 30 '11 at 09:57

gprathour

14,813
5
66
90

score 1 · Answer 3 · answered Nov 30 '11 at 09:58

1

use unix command 'tail', the option '-f' and '-F' is for the same command is very handy as well.

See here http://www.thegeekstuff.com/2009/08/10-awesome-examples-for-viewing-huge-log-files-in-unix/ for examples or just google around for examples.

answered Nov 30 '11 at 09:58

Samyak Bhuta

1,256
7
20

I am generating log file using log4j and i have to read it in java. – Rookie Nov 30 '11 at 10:05
Seems to be similar : http://stackoverflow.com/questions/557844/java-io-implementation-of-unix-linux-tail-f – Nehal Dattani Nov 30 '11 at 10:18

Costis Aivalis · Answer 4 · 2011-11-30T10:38:25.657

RandomAccessFile is a good option. If you leave the application you will have to persist the place of your last read before leaving, in order to avoid rereading information.

Log files, on the other hand, tend to become quite large for heavy event flow. Rotating log files will allow you to shift your problem a little towards file naming. Your can configure your system to produce one log file per day like here:

app_access.2011-11-28.log, 
app_access.2011-11-29.log, 
app_access.2011-11-30.log,
...

If the files you get are still very large, you may rotate them by date and time and you will have also the hour as part of the file name. Your files could then rotate, let's say, every three hours or even every hour. This will give you more log files to read, but they will be smaller, thus easier to process. The date and time range you want to seek for will be part of the file name.

You could also additionally rotate by file size. If you select a maximum file size you can deal with you could avoid accessing randomly a huge file completely.

Read data from log file periodically

4 Answers4