Return large file from JAX-WS service

Question

I'm trying to return large file from JAX-WS service method, but getting strange client error. Here is my code:

@WebService(targetNamespace = "http://java.CAEServer", portName = "CAEInstance")
public interface Instance  {
    @WebMethod(action = "http://java.CAEServer/getResultsArch")
    DataHandler getResultsArch(org.caebeans.caeserver.Instance instance);
}

Implementation:

MTOM(enabled = true, threshold = 2048)
@BindingType(SOAPBinding.SOAP11HTTP_MTOM_BINDING)
@StreamingAttachment(parseEagerly=true, memoryThreshold=4000L)
@WebService(endpointInterface = "org.caebeans.wsrf.Instance")
public class InstanceImpl implements Instance {
    @Override
    public DataHandler getResultsArch(org.caebeans.caeserver.Instance instance) {
        try {
            return workStorageManager.getWorkPackage(instance.getId());
        } catch (Exception e) {
            logger.fatal("Failed to zip work", e);
            throw new RuntimeException("Failed to zip results");
        }
    }
}

getWorkArch returns DataHandler with zipped data. Here is client code:

Instance instanceTransport = new InstanceImplService().getInstanceImplPort();
SOAPBinding binding = (SOAPBinding) ((BindingProvider) instanceTransport).getBinding();
binding.setMTOMEnabled(true);

byte[] resultArch = instanceTransport.getResultsArch(instance);

And when I'm trying to run it, I'm getting errors. Here is server stack trace:

java.io.IOException: Broken pipe
at sun.nio.ch.FileDispatcher.write0(Native Method)
at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:29)
at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:69)
at sun.nio.ch.IOUtil.write(IOUtil.java:40)
at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:336)
at sun.net.httpserver.Request$WriteStream.write(Request.java:397)
at sun.net.httpserver.ChunkedOutputStream.writeChunk(ChunkedOutputStream.java:108)
at sun.net.httpserver.ChunkedOutputStream.write(ChunkedOutputStream.java:77)
at sun.net.httpserver.PlaceholderOutputStream.write(ExchangeImpl.java:390)
at com.sun.xml.internal.ws.transport.http.server.ServerConnectionImpl$2.write(ServerConnectionImpl.java:163)
at javax.activation.DataHandler.writeTo(DataHandler.java:294)
at com.sun.xml.internal.ws.encoding.MtomCodec$ByteArrayBuffer.write(MtomCodec.java:189)
at com.sun.xml.internal.ws.encoding.MtomCodec.encode(MtomCodec.java:156)
at com.sun.xml.internal.ws.encoding.SOAPBindingCodec.encode(SOAPBindingCodec.java:249)
at com.sun.xml.internal.ws.transport.http.HttpAdapter.encodePacket(HttpAdapter.java:328)
at com.sun.xml.internal.ws.transport.http.HttpAdapter.access$100(HttpAdapter.java:82)
at com.sun.xml.internal.ws.transport.http.HttpAdapter$HttpToolkit.handle(HttpAdapter.java:470)
at com.sun.xml.internal.ws.transport.http.HttpAdapter.handle(HttpAdapter.java:233)
at com.sun.xml.internal.ws.transport.http.server.WSHttpHandler.handleExchange(WSHttpHandler.java:95)
at com.sun.xml.internal.ws.transport.http.server.WSHttpHandler.handle(WSHttpHandler.java:80)
at com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:65)
at sun.net.httpserver.AuthFilter.doFilter(AuthFilter.java:65)
at com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:68)
at sun.net.httpserver.ServerImpl$Exchange$LinkHandler.handle(ServerImpl.java:557)
at com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:65)
at sun.net.httpserver.ServerImpl$Exchange.run(ServerImpl.java:529)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)

And here is client stack trace:

xception in thread "main" java.lang.OutOfMemoryError: Java heap space
at com.sun.xml.internal.org.jvnet.staxex.ByteArrayOutputStreamEx.readFrom(ByteArrayOutputStreamEx.java:60)
at com.sun.xml.internal.org.jvnet.staxex.Base64Data.get(Base64Data.java:225)
at com.sun.xml.internal.org.jvnet.staxex.Base64Data.length(Base64Data.java:266)
at com.sun.xml.internal.ws.encoding.MtomCodec$MtomXMLStreamReaderEx.getTextCharacters(MtomCodec.java:508)
at com.sun.xml.internal.bind.v2.runtime.unmarshaller.StAXStreamConnector.handleCharacters(StAXStreamConnector.java:312)
at com.sun.xml.internal.bind.v2.runtime.unmarshaller.StAXStreamConnector.bridge(StAXStreamConnector.java:176)
at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal0(UnmarshallerImpl.java:351)
at com.sun.xml.internal.bind.v2.runtime.BridgeImpl.unmarshal(BridgeImpl.java:109)
at com.sun.xml.internal.bind.api.Bridge.unmarshal(Bridge.java:222)
at com.sun.xml.internal.ws.client.sei.ResponseBuilder$DocLit.readResponse(ResponseBuilder.java:514)
at com.sun.xml.internal.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:110)
at com.sun.xml.internal.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:78)
at com.sun.xml.internal.ws.client.sei.SEIStub.invoke(SEIStub.java:107)
at $Proxy31.getResultsArch(Unknown Source)
at Main.main(Main.java:73)

What's wrong?

EDIT: Here is hosting code in my Main class:

Endpoint instanceEndpoint = Endpoint.publish(serverHost + INSTANCE_PATH, instance);

Client is runned with default MacOS java 6 settings. I'm trying to download 400Mb file, and without TOM enabled, the server fell with OutOfMemory error.

What is the JAX-WS runtime? What about the memory for your client? The byte array may constains the full file? — Paul Vargas, Mar 11 '13 at 14:54
I always like to verify that it is using MTOM using Wireshark, due to client and server having to agree to use it. As suggested below, you can also tell the client to use more heap space. — davidfmatheson, Mar 11 '13 at 15:38

Santosh · Answer 1 · 2013-03-11T15:42:15.970

3

Here are some suggestions and observations:

Broken pipe exception is thrown when the server is responding to the client but client connection terminates midway.
This is quite obvious as your client crashed with out of memory error and its not able to handle the amount of data received from the server with the given heap-size (I see you are trying to hold the data in byte array.).
One and quickest solution is to increase the heap-size of the client VM to appropriate value.
Other and more practical option (provided your use case entertains this) is to stream the data in a persistent location (e.g. a file). This way the entire data will not be loaded in the memory

EDIT:

If you check the documentation. There a StreamingDataHandler class which can be used to stream the data. This this particular link. There is a subtopic called Streaming SOAP Attachments. It gives and example of large file download, very close to your use case. I think you only need to change the client side code and use StreamingDataHandler instead of DataHandler

edited Mar 11 '13 at 15:42

answered Mar 11 '13 at 15:25

Santosh

17,667
4
54
79

Can you provide some streaming examples? I've tryed to do this in these code samples. Also, file size can be various - even several gygabytes. – skayred Mar 11 '13 at 15:28
I've changed my code as written in the article, but still getting OutOfMemory on client :( – skayred Mar 12 '13 at 13:41
Can you please tell on which line OME occurs ? – Santosh Mar 13 '13 at 02:45
It occurs on the line with remote call (in my case - `getResultArch`). Also, looks like it working on Linux, but not MacOS – skayred Mar 13 '13 at 03:38
The default heap size allocated to VM differs from platform to platform. You can [Check the heapsizes](http://stackoverflow.com/questions/2015463/how-to-view-the-current-heap-size-that-an-applicatin-is-using) allocated on different machine and find out the precise memory requirement. – Santosh Mar 13 '13 at 06:21

Return large file from JAX-WS service

1 Answers1