Stream xml input to sax Parser, How to print the xml streamed?

Question

Well I am trying to connect to one remote server via socket, and I get big xml responses back from socket, delimited by a '\n' character.

<?xml version="1.0" encoding="UTF-8"?>
<Response>
    <data>
       .......
       .......
    </data>
</Response>\n   <---- \n acts as delimiter 
<?xml version="1.0" encoding="UTF-8"?>
<Response>
    <data>
        ....
        ....
    </data>
</Response>\n
..

I am trying to parse these xml using SAX Parser. Ideally I want to get one full response to a string by searching for '\n' and give this response to parser. But since my single response is very large, I am getting outOfMemory Exception when holding such a large xml in string..So the only option remained was to stream the xml to SAX.

SAXParserFactory spfactory = SAXParserFactory.newInstance();
SAXParser saxParser = spfactory.newSAXParser();
XMLReader xmlReader = saxParser.getXMLReader();

xmlReader.setContentHandler(new MyDefaultHandler(context));

InputSource xmlInputSource  =   new InputSource(new    
                    CloseShieldInputStream(mySocket.getInputStream()));
xmlReader.parse(xmlInputSource);

I am using closeShieldInputStream to prevent SAX closing my socket stream on exception because of '\n'. I asked a previous question on that..

Now sometimes I am getting Parse Error

org.apache.harmony.xml.ExpatParser$ParseException: At line 1, column 8: not well-formed (invalid token)

I searched for it and found out this error normally comes when the encoding of actual xml is not same as what SAX expecting. I wrote a C program and print out the xml, and all my xml is encoded by UTF-8.

Now my question..

Is there any other reason for the above given error in xml parsing other than encoding issue
Is there any way to print (or write to any file) the input to SAX as it streams from socket?

After trying Hemal Pandya's answer..

OutputStream log = new BufferedOutputStream(new FileOutputStream("log.txt"));
InputSource xmlInputSource  =   new InputSource(new CloseShieldInputStream(new   
                                    TeeInputStream(mReadStream, log)));
xmlReader.parse(xmlInputSource);

a new file with name log.txt getting created when I mount the SDCard but it is empty..Am I using this right?

Well Finally how I done it..

I worked it out with TeeInputStream itself..thanks Hemal Pandya for suggesting that..

//open a log file in append mode..
OutputStream log = new BufferedOutputStream(new FileOutputStream("log.txt",true));
InputSource xmlInputSource  =   new InputSource(new CloseShieldInputStream(new   
                                        TeeInputStream(mReadStream, log)));
try{
  xmlReader.parse(xmlInputSource);
  //flush content in the log stream to file..this code only executes if parsing completed successfully 
  log.flush();
}catch(SaxException e){
  //we want to get the log even if parsing failed..So we are making sure we get the log in either case..
  log.flush();
}

Please see my edit, I have added a doctype element to each response..Is that the reason for first error? — Krishnabhadra, Oct 05 '11 at 10:47
Instead of calling `log.flush()` in both `try` and `catch` blocks another option is to do it outisde -- `try{ xmlReader.parse(xmlInputSource); }catch(SaxException e){ /* log exception */ } log.flush(); ` — Miserable Variable, Oct 10 '11 at 17:54
hmmm...But in my case, when I have an exception i am returning right from the exception catch itself..So that is not possible in my case — Krishnabhadra, Oct 11 '11 at 02:54

score 2 · Accepted Answer · edited Oct 04 '11 at 11:13

2

Is there any way to print (or write to any file) the input to SAX as it streams from socket?

Apache Commons has a TeeInputStream that should be useful.

OutputStream log = new BufferedOutputStream(new FileOutputtStream("response.xml"));
InputSource xmlInputSource  =   new InputSource(new    
                    CloseShieldInputStream(new TeeInputStream(mySocket.getInputStream(), log)));

I haven't used it, you might want to try it first in a standalone program to figure out close semantics, though looking at docs and your requirements it looks like you would want to close it separately at end.

edited Oct 04 '11 at 11:13

Krishnabhadra

34,169
30
118
167

answered Oct 04 '11 at 11:08

Miserable Variable

28,432
15
72
133

Thanks for the response..Can you please see the edit in my question – Krishnabhadra Oct 04 '11 at 11:37
As I said I haven't tried it and you should try it in a small standalone program, for example instead of using `socket.getInputStream()` use a `FileInputStream` and instead of using creating an `InputSource` out of it just print it on console. But...are you calling `close()` on it at the end? That could well be a reason it is not getting anything (its buffered). – Miserable Variable Oct 04 '11 at 11:58

score 0 · Answer 2 · answered Oct 04 '11 at 20:23

I am not familiar with Expat, but to accomplish you are are describing in general, you need a SAX parser that supports pushing data into the parser instead of having the parser pull data from a source. Check if Expat supports a push model. If it does, then you can simply read a chunk of data from the socket, push it to the parser, and it will parse whatever it can from the chuck, caching any remaining data for use during the next push. Repeat as needed until you are ready to close the socket connection. In this model, the \n separator would get treated as miscellaneous whitespace between nodes, so you have to use the SAX events to detect when a new <Response> node opens and closes. Also, because you are receiving multiple <Response> nodes in the data, and XML does not allow more than 1 top-level document node, you would need to push a custom opening tag into the parser before you then start pushing the socket data into the parser. The custom opening tag will then become the top-level document node, and the <Response> nodes will be children of it.

Stream xml input to sax Parser, How to print the xml streamed?

After trying Hemal Pandya's answer..

Well Finally how I done it..

2 Answers2