8

I am using Java on Ubuntu 16.04. Recently I upgraded to Open JDK java version "1.8.0_161" installed using the oracle-java8-installer package (package version 8u161-1~webupd8~0). Since doing this upgrade , I am getting new exceptions when doing JAXB marshalling of Java objects.

Specifically, when attempting to use JAXB to marshal a Java object to XML I get the following exception if the Java object has a String property that contains any newline ("\n") characters and that String property is being serialized as element content in the XML. (As an aside, if the String property is serialized as attribute content, any newline character in the value of the String is converted to a space character and the exception is not triggered.)

What appears to be happening is that

com.sun.xml.internal.bind.v2.runtime.output.XMLStreamWriterOutput$NewLineEscapeHandler.escape

converts the newline character in the String property of the Java object to the entity reference 
. This entity reference is then written out to the XML output stream but when verifying the entity reference name, the exception is being thrown because #xa is not being recognised as a valid entity reference name.

Is this the expected behaviour? If so, what should I do to preserve the newline characters in the serialization of the Java object? If not, what should I do to work around this problem?

The relevant part of the stack trace is:

... Caused by: javax.xml.stream.XMLStreamException: Invalid name start character '#' (code 35) (name "#xa")
at com.fasterxml.aalto.out.XmlWriter.throwOutputError(XmlWriter.java:472)
at com.fasterxml.aalto.out.XmlWriter.reportNwfName(XmlWriter.java:383)
at com.fasterxml.aalto.out.ByteXmlWriter.verifyNameComponent(ByteXmlWriter.java:235)
at com.fasterxml.aalto.out.ByteXmlWriter.constructName(ByteXmlWriter.java:181)
at com.fasterxml.aalto.out.WNameTable.findSymbol(WNameTable.java:324)
at com.fasterxml.aalto.out.StreamWriterBase.writeEntityRef(StreamWriterBase.java:615)
at net.galexy.fieldguide.jaxb.CustomXMLStreamWriter.writeEntityRef(CustomXMLStreamWriter.java:198)
at com.sun.xml.internal.bind.v2.runtime.output.XMLStreamWriterOutput$XmlStreamOutWriterAdapter.writeEntityRef(XMLStreamWriterOutput.java:277)
at com.sun.xml.internal.bind.v2.runtime.output.XMLStreamWriterOutput$NewLineEscapeHandler.escape(XMLStreamWriterOutput.java:242)
... 60 more

For example, if I unmarshall the following XML:

<?xml version='1.0' encoding='UTF-8'?>
<description>
   <note>The text of the note</note>
</description>

and then attempt to marshall it back to XML then no exception is thrown.

If, however, there is a new line in the middle of the note content:

<?xml version='1.0' encoding='UTF-8'?>
<description>
   <note>The text of
         the note</note>
</description>

Then the exception is thrown.

The JAXB context that is being used is com.sun.xml.internal.bind.v2.runtime.JAXBContextImpl.

The JAXB marshaller that is being used is com.sun.xml.internal.bind.v2.runtime.MarshallerImpl

Looking for more information on the changes, I came across the following bug report that suggests others have encountered the same change with this release of JAXB:

JDK-8196491 Newlines in JAXB string values of SOAP-requests are escaped to "&#xa;"

The answer to this stack overflow question suggests that I can resume control over character escaping by getting my marshaller to use a custom implementation of com.sun.xml.bind.marshaller.CharacterEscapeHandler.

That is puzzling me because javax.xml.bind.Marshaller does not appear to declare a static property name com.sun.xml.bind.marshaller.CharacterEscapeHandler while it does declare other property names like Marshaller.JAXB_FORMATTED_OUTPUT, which equals "jaxb.formatted.output.

Even if I could instruct the marshaller to use my custom character escape handler, I am not totally sure what I should be doing within that escape handler. Is there an appropriate base escape handler that I can override to inherit all of the standard escape handling which ensuring that I intervene to stop escaping of the newline characters?

I have also tried Oracle Java 9 (package version 9.0.4-1~webupd8~0) and that version of Java has the same issues.

I have also tried the next release of Oracle Java 8 (1.8.0_162) and that version has the same issues.

Downloading an older version of Java from the Oracle website (1.8.0_152) sorts out the problem but is not a satisfactory way of resolving the problem.

Geoff S
  • 175
  • 2
  • 13

2 Answers2

3

In my case, I'm using JAXB to convert a few objects into XML and serialise them to a file, via StAX/WoodStox. I've managed to fix the problem at issue by filtering the XML that is being serialised. In detail, the approach is like:

  1. Define a custom StreamWriter2Delegate, override writeEntityRef(), so that, when this method receives the wrong entity code (#xd or #xa), it invokes its delegate to actually write back the original character (i.e., \n or \r), which doesn't actually need to be escaped:

    @Override
public void writeEntityRef ( String eref ) throws XMLStreamException
{
    if ( eref == null || !eref.startsWith ( "#x" ) ) {
        super.writeEntityRef ( eref );
        return;
    }
    String hex = eref.substring ( 2 );
    for ( char c: new char[] { '\r', '\n' } )
        if ( Integer.toHexString ( c ).equals ( hex ) ) {
            this.writeCharacters ( Character.toString ( c ) );
            return;
    }
    super.writeEntityRef ( eref );
}

This is equivalent (apart from some overhead) to the fix they've already filed for this problem, which should be available with JDK8u192 (and should already be in JDK 9/10).

  1. Wrap your XMLStreamWriter2 with the above filter, for instance:

    FileOutputStream fout = new FileOutputStream ( "test.xml" );
WstxOutputFactory wsof = (WstxOutputFactory) WstxOutputFactory.newInstance();
XMLStreamWriter2 xmlOut = (XMLStreamWriter2) wsof.createXMLStreamWriter ( fout, CharsetNames.CS_UTF8 );
xmlOut = new NewLineFixWriterFilter ( xmlOut );
// Now write into xmlOut, directly or via JAXB

The complete/production code is here. It shouldn't be difficult to adapt the same approach to similar pipelines (in general, the problem occurs because com.sun.xml.internal.bind.v2.runtime.output.XMLStreamWriterOutput escapes \n and \r the wrong way, so the trick is to hijack this wrong encoding from the upper levels).

zakmck
  • 2,715
  • 1
  • 37
  • 53
2

Geoff S,

I tried to comment on the existing post but I quickly found out that you need to have “50 reputations” which I do not have.

It appears that I am experiencing a similar issue when we moved to JDK 1.8.0_161 and 1.8.0_162 some of our SOAP services started throwing the exceptions below

Feb 28, 2018 8:34:12 AM com.sun.xml.internal.messaging.saaj.soap.SOAPDocumentImpl createEntityReference
SEVERE: SAAJ0543: Entity References are not allowed in SOAP documents
SEVERE: java.lang.UnsupportedOperationException: Entity References are not allowed in SOAP documents
javax.xml.ws.WebServiceException: java.lang.UnsupportedOperationException: Entity References are not allowed in SOAP documents
    at com.sun.xml.internal.ws.handler.ClientSOAPHandlerTube.callHandlersOnRequest(ClientSOAPHandlerTube.java:135)
    at com.sun.xml.internal.ws.handler.HandlerTube.processRequest(HandlerTube.java:112)
    at com.sun.xml.internal.ws.api.pipe.Fiber.__doRun(Fiber.java:1121)
    at com.sun.xml.internal.ws.api.pipe.Fiber._doRun(Fiber.java:1035)
    at com.sun.xml.internal.ws.api.pipe.Fiber.doRun(Fiber.java:1004)
    at com.sun.xml.internal.ws.api.pipe.Fiber.runSync(Fiber.java:862)
    at com.sun.xml.internal.ws.client.Stub.process(Stub.java:448)
    at com.sun.xml.internal.ws.client.sei.SEIStub.doProcess(SEIStub.java:178)
    at com.sun.xml.internal.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:93)
    at com.sun.xml.internal.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:77)
    at com.sun.xml.internal.ws.client.sei.SEIStub.invoke(SEIStub.java:147)
    at com.sun.proxy.$Proxy38.getUserProfile(Unknown Source)

As indicated by the above question and other threads:

It has something to do with newlines in the payload. For example some of our payloads include XML strings that have new lines which cause the issue. however if the newlines are removed prior to calling the service then it works. See immediately below:

Fail

<?xml version="1.0" encoding="UTF-8"?>
<user>
<userId>XXXX</userId>
<name>XXXXXX, XXXXXX</name>
<phone>(xxx)xxx-xxxx</phone>
<title><![CDATA[MY TITLE]]></title>
<mail>xxx@xxxx.com</mail>
</user>

Works

<?xml version="1.0" encoding="UTF-8"?><user><userId>XXXX</userId><name>XXXXXX, XXXXXX</name><phone>(xxx)xxx-xxxx</phone><title><![CDATA[MY TITLE]]></title><mail>xxx@xxxx.com</mail></user>

Do you or anyone else know if there is workaround other than stripping the payload from “new lines”, and is this considered a bug in the latest Oracle JDK and are there any plans to rectify the behavior.

Thanks

max

Massimo Mirabito
  • 21
  • 3
  • Hi Max, my workaround was to manually install a previous version of the Java JDK. It is not clear to me at this stage whether this is deemed a bug. – Geoff S Mar 01 '18 at 23:20
  • Hi Geoff, thanks for the response, we did the same thing by downgrading I am just not sure how long security will allow us. I was hoping that someone from Oracle would jump on ther thread a provide insight and see if this a bug that they plan to fix or just let it be. thanks max – Massimo Mirabito Mar 02 '18 at 15:50
  • which version? Can you provide link? – Mateusz Gebroski Sep 01 '20 at 10:16