1

I have WS where in the SOAP payload there is a xsd:base64Binary element.

Initially this was designed for max 1MB but the requirement changed and now it should accept 50 MB uploaded files converted to base64 :( In java this becomes HUGE in terms of memory.

My theoretical solution is to stream from one end to the other these 'attachments' (files are uploaded from the web app, then converted to base64 and a web service is called to store in some Adobe app) - very inefficient i say

I read about Stax but that is not for WebServices.

Is there a way to use streaming in combination with Webservices ?

( MTOM ,but did not find a sample with streaming)

GEOCHET
  • 21,119
  • 15
  • 74
  • 98
Cris
  • 4,947
  • 6
  • 44
  • 73
  • 1
    Take a look at http://jax-ws.java.net/nonav/jax-ws-20-fcs/arch/com/sun/xml/ws/developer/StreamingDataHandler.html and http://stackoverflow.com/a/870721/169277 – ant Jul 19 '12 at 20:42
  • 1
    Instead of streaming a large document, you will want to use the JAX-WS mechanisms to send the binary data as attachments. – bdoughan Jul 19 '12 at 20:50
  • Thanks to both...indeed nice directions – Cris Jul 19 '12 at 21:36
  • @bdoughan Would sending the binary data as attachment prevent loading the entire data in memory on the client side before sending? – yolob 21 Apr 28 '20 at 14:19

1 Answers1

3

I think you could do this with a custom servlet and overriding the post method.

Read enough of the request's InputStream through Stax to find the element containing the base64, then pipe it to some sort of Base64InputStream until you reach the end of the element. Use Stax or JaxB to transform an object into a response and send it back.

Whatever you do, don't buffer up the entire request. Don't use a typical soap framework like CXF or JAXWS without thinking about how it handles buffering. You're going to hit your garbage collector so hard it'll panic, run away, never call you again.

EDIT: Garbage collection

If you're running a 4gb heap, and assuming you're using HotSpot, make sure you're using the latest Java from Oracle (7u5). Next, switch to a collector that's meant to handle large heaps: either g1gc (-XX:+UseG1GC) or CMS (-XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode)

In addition, consider adding the following:

-XX:+OptimizeStringConcat -XX:+AggressiveOpts -XX:+UseFastAccessorMethods -XX:+UseLargePages -XX:+UseStringCache -XX:+UseCompressedOops

Community
  • 1
  • 1
Jonathan S. Fisher
  • 8,189
  • 6
  • 46
  • 84
  • indeed ...the server now has 4gb...but gc takes more then 1 min.Anyway...when gc is in action why the whole JVM freeze ? – Cris Jul 19 '12 at 21:31
  • The user threads in the JVM are frozen because the garbage collector you are using is a "Stop the world" design. – Jonathan S. Fisher Jul 19 '12 at 22:06