How to redirect outputs of invoked commands/programs to log4j?

Question

I write a program and handle my logs with a logging tool (e.g., Java's log4j or Python's logging), so all logs produced by myself can go to a log file managed by the logging tool.

I'd also invoke a command or a third party program in my program, which writes all its outputs to the console by default. How can I redirect those outputs to the log file managed by the logging tool (and make them conform to the logging format if possible)?

score 0 · Answer 1 · answered Sep 01 '15 at 07:58

0

In java, you could use IoBuilder from log4j to build your PrintStream. IoBuilder is included in Apache's Log4j Streaming Interface, an addition to Log4J.

Once you have your PrintStream, you can set the system's default PrintStream...

 IoBuilder builder = IoBuilder.forLogger(MyClass.class);
 PrintStream myPrintStream = builder.buildPrintStream();
 System.out = myPrintStream;

This way, if other library goes System.out.print() or println() it will be logged through your logger, with your logger format.

answered Sep 01 '15 at 07:58

Ferdinand Neman

680
3
6

It won't have any effect on the output of an external process ("third party program"). You would need something that redirects at a file descriptor level instead. Here's a [code example in Python](http://stackoverflow.com/a/22434262/4279) – jfs Sep 01 '15 at 09:31

score 0 · Accepted Answer · answered Sep 01 '15 at 09:48

To redirect all standard output of an external process to a file in Python:

#!/usr/bin/env python
from subprocess import check_call, STDOUT

with open('log', 'ab', 0) as file:
    check_call(['program', 'arg 1', 'arg 2'], stdout=file, stderr=STDOUT)

The output is redirected as is. To make it conform to the logging format, you might need it to pass through your program explicitly:

#!/usr/bin/env python3
import logging
from subprocess import Popen, PIPE, STDOUT

logging.basicConfig(filename='log',level=logging.DEBUG)
with Popen(['program', 'arg 1', 'arg 2'], stdout=PIPE, stderr=STDOUT, bufsize=1,
           universal_newlines=True) as process:
    for line in process.stdout:
        logging.debug('program output %s', line.rstrip('\n'))

The code decodes program's standard output using locale.getpreferredencoding(False) and appends lines to the log file using logging module (you could configure whatever logging format you like using standard logging tools).

In Python I'd like to use a rotating file logger in which case I needn't open a file explicitly as *with open('log', 'ab', 0) as file ....* . I hope to redirect all outputs of an external process to the logging module and let the logging module do everything for me. How can we do that? — Tao Xiao, Sep 02 '15 at 00:53

How to redirect outputs of invoked commands/programs to log4j?

2 Answers2