Java logging using getHead() with append

Question

We're developing a web application and I'm implementing event logging on the server side. I decided a nice approach would be to extend the Java logging API to log events to a CSV file. I created a java.util.logging.Formatter class as follows:

public class EventLogCsvFormatter extends Formatter {
private static final SimpleDateFormat SDF = new SimpleDateFormat("MM/dd/yyyy h:mm a");
private static final String[] COL_HEADERS = {"Date/Time", "Source", "Event Type", "Application Context", "Description", "Fields"};

public EventLogCsvFormatter() {
}

@Override
public String getHead(Handler h) {
    return String.join(",", COL_HEADERS);
}

@Override
public String format(LogRecord record) {
    if (record instanceof EventLogRecord) {
        EventLogRecord customLogRecord = (EventLogRecord) record;
        String[] fields = customLogRecord.getFields();
        List<String> textList = new ArrayList<>();

        textList.add(SDF.format(new Date(record.getMillis())));
        textList.add(customLogRecord.getSource() != null ? customLogRecord.getSource() : "Not Given");
        textList.add(customLogRecord.getEventType() != null ? customLogRecord.getEventType().toString() : "Not Given");
        textList.add(customLogRecord.getEventContext() != null ? customLogRecord.getEventContext().toString() : "Not Given");
        textList.add(customLogRecord.getMessage());

        if (fields != null && fields.length > 0) {
            for (String field : fields) {
                textList.add(field);
            }
        }

        String retVal = "\n" + String.join(",", textList);
        return retVal;
    }

    return "";
}

}

The idea to implement the getHead() method is to provide the column headers for each CSV file. Note that Java's own XMLFormatter does something similar in returning the XML file header string in its getHead() method. As it turns out, Java logging does not take into account the append flag when calling getHead(). The append flag basically tells the logger to re-open the existing file and continue logging to it upon startup. Because our test server gets bounced quite a bit, most of the CSV files generated have the column header names in multiple places within the files, not just at the top.

I don't think there's any way around this, since pretty much all of Java logging handler code has private or package scoped fields and methods. So, I can't even write my own custom handler to work here. Is this a bug and I'm SOL? Should I just go ahead and leverage a different logging API (e.g. Log4J)?

Answer 1

The FileHandler RFEs related to this problem are JDK-4629315: Appending of XML Logfiles doesn't merge new records and JDK-5036335: Provide method to obtain log file name(s) and path(s).

The trick to making this work is that you have to be able to query the current log file when header is requested. If the current log file length is zero then the formatter has to return headers.

In your custom formatter you can have the getHead method try to locate the open file and query the length using the either java.io or java.nio.

@Override
public String getHead(Handler h) {
    boolean writeHeader = true;
    try {
        if (h instanceof FileHandler) {
            writeHeader = lengthOpen((FileHandler) h).longValue() == 0L;
        }
    } catch (SecurityException ignore) {
    }

    if (writeHeader) {
        return ""; //TODO: Insert your CSV headers.
    } else {
        return super.getHead(h); //Skip headers.
    }
}

private Number lengthOpen(Handler h) {
    if (h instanceof FileHandler) {
        String p = h.getClass().getName();
        LogManager manager = LogManager.getLogManager();
        p = manager.getProperty(p.concat(".pattern"));
        //TODO: Deal with FileHandler patterns.
        if (p != null) {
            File f = new File(p);
            //TODO: Implement file listing and filtering.
            return f.length();
        }
    }
    return 0L;
}

Otherwise, if you want to do some hackery you can resort to using reflection.

@Override
public String getHead(Handler h) {
    boolean writeHeader = true;
    try {
        if (h instanceof FileHandler) {
            writeHeader = lengthFrom((FileHandler) h).longValue() == 0L;
        }
    } catch (SecurityException ignore) {
    }

    if (writeHeader) {
        return ""; //TODO: Insert your CSV headers here.
    } else {
        return super.getHead(h); //Skip headers.
    }
}

private Number lengthFrom(FileHandler h) {
    try {
        Field f = StreamHandler.class.getDeclaredField("output");
        f.setAccessible(true);
        OutputStream out = (OutputStream) f.get(h);
        f = out.getClass().getDeclaredField("written");
        f.setAccessible(true);
        return (Number) f.get(out);
    } catch (ReflectiveOperationException roe) {
        h.getErrorManager().error(null, roe, ErrorManager.FORMAT_FAILURE);
    }
    return 0L;
}