1

Let me begin with a brief explanation of the system I am using:

We have a system that runs as a Daemon, running work-units as they come in. This daemon dynamically creates a new Thread when it is given a new work-unit. For each of these units of work, I need Log4J to create a new log file to append to - the filename will be provided at runtime, when the new log file must be created. This daemon must be able to remain alive indefinitely, which I believe causes some memory concerns, as I will explain.

My first thought was to create a new Logger for each work-unit, naming it after the thread, of course. The work-unit's thread retains a reference to that Logger. When the unit is finished, it will be garbage-collected, but the problem is that Log4J itself retains a reference to the Logger, which will never be used again. It seems likely that all these Loggers will cause the VM to run out of memory.

Another solution: subclass Filter, to filter Appenders by the thread names, and place them on the same Logger. Then, remove the Appenders as the work-units complete. Of course, this necessitates adding the code to remove the appenders. That will be a lot of code changes.

I've looked into NDC and MDC, which appear to be intended for managing interleaved output to the same file. I've considered proposing this as a solution, but I don't think it will be accepted.

I want to say that Log4J appears that it is intended not to function this way, that is, dynamically creating new log files at runtime, as they are required (or desired). So I am not sure which direction to look in next - Is log4j not the solution here, or did I comPLETELY miss something? Have I not looked closely enough at NDC? Or is my concern with Log4J holding onto Loggers a nonissue for reasons I don't see?

tristan.marks
  • 17
  • 5
  • Instead of using as logger to indirectly create a new file each time, you could write to a new file with FileWriter and close it when you have finished with it. – Peter Lawrey Oct 16 '12 at 21:56

2 Answers2

0

You could just create a new log method that wraps the normal log method, and append thread id. Something like the following (oversimplified, but you get the idea). Log4j is already thread save I believe, so as long as you aren't logging a ton, you should be fine. Then you can easily grep on thread id.

public log(long id, String message)
{
   logger.log("ThreadId: id + "message: " + message);
}
aglassman
  • 2,643
  • 1
  • 17
  • 30
  • Thank you for your reply. That would log the output for all the threads to the same file, correct? That isn't the solution I am looking for. I'm not looking to ensure thread safety; I know Log4J does that. – tristan.marks Oct 16 '12 at 23:03
  • Check the links supplied in this question. Looks like they were trying to do the same, but with requests rather than threads. http://stackoverflow.com/questions/2331128/log4j-one-log-file-per-request?rq=1 – aglassman Oct 17 '12 at 18:11
0

ThreadLocal appears to be the solution for you.

Njax3SmmM2x2a0Zf7Hpd
  • 1,354
  • 4
  • 22
  • 44