Commons Logging priority best practices

Question

This may be a purely subjective question (if no organization has attempted to standardize this), but my team struggles with this more than you would think.

We use Apache Commons Logging as our logging interface and often the use of priority is not consistent across our development team. For example, some developers log any caught exception to fatal (log.fatal(message)) even though the flow is able to handle the error whereas others only log to fatal when something cause the program to necessarily cease execution for whatever reason.

I would like to know how other teams define each priority. Does anyone work at a company that explicitly tries to define best practices for this? Has Jakarta weighed in on this?

My goal would be to send a simple recommendation for each priority out to my whole team so that we can more effectively handle our unwieldy application logging in a consistent fashion.

Answer 1

Here's what we use (and I'd say many others as well):

• FATAL: errors that endanger the consistency of the system and might require an immediate shutdown/restart - very rarely manually used
• ERROR: exceptions that should not be thrown and represent a bug in the system (normally all exceptions that are not caught up to a certain point)
• WARN: exceptions that might occur but are accounted for or situations that might imply a logic/usage error - we decide whether to track those or not
• INFO: whatever you need to have in the log, e.g. what users currently do (in our case of web applications: what page the user is navigating to etc.)
• DEBUG: debug only messages like timings etc, generally turned off in the logs
• TRACE: we don't use that, you might use it for even more specific debugging info

In some cases we start with logging messages as ERROR, in order to get the notification when something we'd normally not like to happen occurs. Later, if we decide that the source for that error can not be removed, we handle it and change the log level to WARN.

Note that our test and production systems are set up to send an email for FATAL and ERROR. Thus we don't have to check the log files for those manually but only have to check one email inbox.

Hope that helps.

Edit: did you already have a look at the Apache Commons Logging best pratices?

Answer 2

I have always used this as a guideline; I use Log4j more than Commons Logging but this may still be helpful:

DEBUG - for genuinely debug-level info; will not be seen in production or shipped product, as INFO will be the minimum level; good for capturing timings, number of occurrences of events, etc

INFO - minimum level for production/shipped usage; record data likely to be useful in forensic investigations and to confirm successful outcomes ("stored 999 items in DB OK"); all info here must be such that you would be OK with end users/customers seeing it and sending you it, if need be (no secrets, no profanity!)

WARN - not an error level as such, but useful to know the system may be entering dodgy territory, e.g. business logic stuff like "number of ordered products < 0" which suggests a bug somewhere, but isn't a system exception; I tend not to use it that much to be honest, finding things tend to be more natural fits to INFO or ERROR

ERROR - use this for exceptions (unless there's a good reason to reduce to WARN or INFO); log full stacktraces along with important variable values without which diagnosis is impossible; use only for app/system errors, not bad business logic circumstances

FATAL - only use this for an error of such high severity that it literally prevents the app from starting / continuing

Additionally - review your logging often, both with DEBUG and INFO levels active, you want to be able to follow a good narrative through, with prominent events easy to find, and to make sure you're not doing anything too verbose which detracts from readability.

Also ensure you use a pattern which leads to neat columns for things like timestamps, source-classes and severity - again, it helps readability massively.

Answer 3

My take

• Fatal: The program is in an state that cannot be handled and must be terminated (automatically or by the user).
• Error: The operation of the program failed in a way detectable by the user (the changes were not saved / a file could not be read), but the program still can work (try to load another file).
• Warning: Something did not go as planed but the user did not notice it (a server did not answer a ping; perhaps when that server is needed it will cause an error/fatal)
• Info: User actions / major program actions (file loaded, automatic backup stored).
• Debug: Tracing information. What part of the program is running, which are the values of the parameters

Answer 4

This is what my company recommends:

TRACE - Messages probably only useful during the development cycle, and possibly generated far too frequently for suitable use in production. For example, if I were logging intermediate values in an inner loop, I’d use TRACE.

DEBUG - Messages used to log various steps in the normal running of the server. Typically these would be aimed at developers rather than operations staff.

INFO - Messages of a positive or neutral nature which we would expect to be logged in a production environment. Should be meaningful to operations staff.

WARN - Messages indicating a condition possibly endangering the ability of the server to respond to requests in an accurate and timely fashion.

ERROR - Messages indicating an unexpected behaviour or condition.

FATAL - Messages indicating an unexpected behaviour or condition which leaves the continued running of the application process impossible or dangerous.

We expect the logs in production to be set at INFO, and we expect them to be read by people other than the developers. Style of log messages is a whole other conversation...

Answer 5

If you are looking for a simple recommendation that is supported by the industry, why not just look at the industry simple implementation/documentation?

We use a logback/log4j API as a logging level guide => which makes it simple / documented / supported by the industry:

• Level ALL has the lowest possible rank and is intended to turn on all logging.

• Level DEBUG designates fine-grained informational events that are most useful to debug an application.

• Level ERROR designates error events that might still allow the application to continue running.

• Level FATAL designates very severe error events that will presumably lead the application to abort.

• Level INFO designates informational messages that highlight the progress of the application at coarse-grained level.

• Level OFF has the highest possible rank and is intended to turn off logging.

• Level TRACE designates finer-grained informational events than the DEBUG

• Level WARN designates potentially harmful situations.

Answer 6

This answer is taken from a post that I wrote about Logging best practices including logging levels that you may find relevant:

In my opinion, it’s sufficient to have: debug, info, warn and error log levels. Some engineers like to have additional levels like verbose, trace, alert, critical, fatal, and some creative developers even invent additional levels…

Please don’t do that. Less is more!

Let’s discuss the different levels:

ERROR - Log entries that indicate a system error that may be critical. For example, an HTTP call failed. A former colleague of mine wrote a nice piece that includes a section on logging exceptions which I recommend reading

WARN - Significant events indicate an error, but the behavior is either expected or not critical. This could be a failed charge where there’s a retry in place, for example.

INFO - Useful information related to the operation of the system. Our production systems are tuned to report this as the minimum level logged.

DEBUG - Intended to be used in pre-prod environments or locally. Use this log level when you don’t want to remove a log-print when merging (because it may prove useful), but also don’t want it to show up in production. There should be a flag/environment-variable that can turn on debug printing (even in production) but it should be used scarcely.

Answer 7

I'm using a simpler approach:

• DEBUG: Turned off in production, thus mainly used by the developer to log certain queries, timings, parameter values, etc.

• INFO: Log everything so that you know in retrospect everything to explain how your results were computed and so that you can fix bugs

• ERROR: Everything that needs someones (developer/operations) attention

I don't use WARN, because no-one ever filters all log files for WARN messages. If it is important, then it is ERROR (and someone has to care about it), if not, it is INFO (and no-one is interested in it until there is a problem). Same for FATAL.

I also don't use TRACE. Everything I need to know during development is DEBUG.