Library for parsing log file in Java

Question

I've a log file with the following structure.

unstructured raw text 
unstructured raw text 
..
..
..

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<message>
...
...
</message> 

unstructured raw text 
..
..


<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<message>
...
...
</message> 

unstructured raw text 
..
..

As you can see there are multiple XML documents embedded inside one single log file. I was wondering if there is a generic utility or library that I can reuse here before I start to write something of my own. I need it in Java.

Thanks.

That design is going to make your life far harder than it needs to be. If you want to use XML and the XML toolchain, then you should embrace it. Stop embedding raw text at the top level, and instead make the entire file a single XML document. It will make your parsing life significantly easier. — Hank Gay, Sep 10 '12 at 20:13
@Hank - I'm not writing the file. I'm getting the file from another system. I need to parse it and use some of the information inside the XML nodes in my system. — Soumya Simanta, Sep 10 '12 at 20:18
In that case, good luck. You're basically going to be stuck doing some ugly hacks by hand: looking for the `` lines, then backing up until you find the last closing tag, and deleting/ignoring everything between those two points; that sort of thing. Honestly, if there is *any* way you can get them to fix the file format upstream, it will make your job far easier. — Hank Gay, Sep 10 '12 at 21:13

score 0 · Answer 1 · edited May 23 '17 at 11:49

0

You can use xml parsers that are built-in into java, but you have to give them only XML as input. So you should read parts of file that are XML into String, and then parse them as Strings. If you don't know how to parse Strings as XML, see here: In Java, how do I parse XML as a String instead of a file?

edited May 23 '17 at 11:49

Community

1
1

answered Sep 10 '12 at 20:18

Zvonimir

339
1
7

score 0 · Accepted Answer · answered Sep 10 '12 at 20:42

I would favour one of the StAX based parsers, the Woodstox ones are particularly performant. If you then need to use a different type of XML parser you can shunt the events from the parser to a generator and feed that XML into e.g. a DOM based parser or a SAX based parser (if you are a masochist... since SAX is a pain of a parser to use).

You will have pseudo-code that looks a little like this:

BufferedReader br = ...
XMLInputFactory inputFactory = XMLInputFactory.newInstance();
Pattern startOfXml = Pattern.compile("<\\?xml.*\\?>");
String line;
while (null != (line = br.readLine()) {
    if (startOfXml.matcher(line).matches()) {
        XMLEventReader xr = inputFactory.createXMLEventFactory(br);
        XMLEvent event;
        while (!(event = xr.nextEvent()).isEndDocument()) {
            // do whatever you want with the event
        }
    } else {
        // do whatever you want with the plain-text
    }
}

Some of the StAX parsers in certain modes may object to the isEndDocument() and in that case you will have to count event level parsing the document and break out once you reach the root level end element. Also some parsers may cache a few characters after the end of the document... worst case you just need to catch an exception for a "malformed" document when the parser notices text after the end element

Library for parsing log file in Java

2 Answers2