1

We are using Log4J to writing log files in our batch applications. The log files after a run are always as huge as several giga bytes. This makes it very difficult and time-consuming to open, read or find useful information from them even after they have been divided into smaller (e.g. 500 MB) ones (by using rolling appender).

I think one reason is that we always write long complete English sentences in all logging levels, because we believe that reading a log file should be like reading a story.

Do you have any best practices to deal with this problem? Is using abbreviations (such as 'OK' instead of 'Successfully created ...') at least in DEBUG and TRACE levels a common usage?

chance
  • 6,307
  • 13
  • 46
  • 70
  • http://stackoverflow.com/questions/153524/code-to-logging-ratio http://stackoverflow.com/questions/576185/logging-best-practices here are 2 questions on stackoverflow – kommradHomer Dec 02 '11 at 13:24

1 Answers1

1

Write your log files as Comma-Separated-Values Text.

There are a number of reasons why:

  1. You can open them in the newer versions of Excel, no matter how large they are (well, up to about a million rows/log entries).

  2. You can filter them by time, if you provide a time column

  3. You can search them.

  4. You can filter or sort them by type or severity (if you provide a column containing that information, i.e. Message, Warning, Error, Critical).

With respect to the verbosity, you can provide two columns; a summary description and a detailed description. Excel allows you to hide or delete columns, so this allows users to slice and dice the files any way they want.

In addition, CSV files are machine-readable, so they can potentially be run through a post-processing program for further analysis.

Robert Harvey
  • 178,213
  • 47
  • 333
  • 501