12

At my workplace, we wrote a custom log4j appender that writes log messages to the database (uses a dedicated thread asynchronously, so no performance hit). I prefer it a lot over writing to log files - a database-based log is much more easy to query and analyze.

Is there an open source solution that does this (for log4j specifically, or any other java loggers)?

Some things that our appender has, and I would like to see in an alternative:

  • Logs exceptions (duh!)
  • Database writes are from a separate thread/pool

Our appender supports the following columns, and I would like to see all of them in whatever solution we find.

  • LogId
  • Time
  • message
  • stacktrace
  • process id
  • thread id
  • machine name
  • component
  • Level (debug/info/warn/...)
  • ThreadName
ripper234
  • 222,824
  • 274
  • 634
  • 905

3 Answers3

2

There is also a DBAppender class provided by log4j (log4j requires a specific set of tables to log using this appender). http://logging.apache.org/log4j/companions/receivers/apidocs/org/apache/log4j/db/DBAppender.html

There is an updated non-Apache jdbc logger available here you may also want to try: http://www.dankomannhaupt.de/projects/index.html

Scott
  • 1,728
  • 11
  • 11
  • Does it answer the (revised) requirements? – ripper234 Nov 12 '10 at 13:35
  • Here is the source of the appender: http://svn.apache.org/viewvc/logging/log4j/companions/receivers/trunk/src/main/java/org/apache/log4j/db/DBAppender.java?view=markup – Scott Nov 12 '10 at 16:42
  • process id and machine name can be passed in as properties if nest the DBAppender definition in a RewriteAppender. RewriteAppender allows you to modify the logging event via a RewritePolicy prior to passing the event off to the nested appender. See: http://logging.apache.org/log4j/companions/receivers/apidocs/org/apache/log4j/rewrite/RewriteAppender.html and http://logging.apache.org/log4j/companions/receivers/apidocs/org/apache/log4j/rewrite/RewritePolicy.html – Scott Nov 12 '10 at 16:45
1

Just curious, wouldn't it severely affect the performance of an application hosting such appender? Logging directly into relational database is quite costly even when you do it asynchronously.

Dima
  • 4,068
  • 4
  • 38
  • 47
  • Well, if you turn on DEBUG messages it might affect performance. Still, we do it in batches, so the effect on performance is not huge unless you're really pumping out hundreds of log messages a second. – ripper234 Nov 13 '10 at 08:14
  • Yes, I thought so. Note however that stack traces could be large. If your application is dedicated for log management - direct commits to the database is all right, but if it's supposed to do real work, the hassle with database layer doesn't seem to be a scalable approach IMHO. I don't think there is an open source appender like you are asking, not seen one. There was something named JDBCAppender once, but it didn't look serious. If you would look into a commercial product - look at logfaces, it does exactly this but in a wider scope and decouple application from log management entirely. – Dima Nov 13 '10 at 09:01
  • We have used such an approach in production code for the last three years, in a medium scale system (about 20 machines). Assuming your data tier is "scalable enough" and you don't pump too many log messages (usually INFO & above), I find the advantages outweigh the disadvantages. – ripper234 Jan 05 '11 at 05:22
0

You don't need a custom appender for LOG4J to write to databases. You can use JDBCAppender bundled with Apache's distribution.

According to APACHE's documentation, this API could be replaced in the future. But for now, we use it and it works very well.

Pablo Santa Cruz
  • 176,835
  • 32
  • 241
  • 292