0

I'm trying to create an application that download and uploads large files, so I don't want the file contents to be stored in memory.

On the mvc controller side I'm using an http message converter that converts to / from InputStream

@Override
public InputStream read(Class<? extends InputStream> clazz, HttpInputMessage inputMessage) throws IOException,
        HttpMessageNotReadableException {
    return inputMessage.getBody();
}

@Override
public void write(InputStream t, MediaType contentType, HttpOutputMessage outputMessage) throws IOException,
        HttpMessageNotWritableException {

    try {
        IOUtils.copy(t, outputMessage.getBody());
    } finally {
        IOUtils.closeQuietly(t);
    }
}

This works well on the server side.

On the client (RestTemplate) side I tried to use the same converter, but I got an exception that the stream has been closed (probably closed when the request was completed).

Client side code:

ResponseEntity<InputStream> res = rest.getForEntity(url, InputStream.class);
// res.getBody() is closed

I've also tried to copy the input stream into a buffer and create a new ByteArrayInputStream and return it to the RestTemplate client and it worked well, however it does require that the data will be read into memory which doesn't suite my demands.

My question is how to keep the stream open until I process it without having to read it all into memory / file?

Any idea will be appreciated.

Regards, Shay

shayma
  • 1
  • 1
  • 2
  • 1
    You need to be inputing and outputing without actually saving anything, this page I would think contains most of the ways of doing this: http://stackoverflow.com/questions/43157/easy-way-to-write-contents-of-a-java-inputstream-to-an-outputstream – mwarren Nov 03 '14 at 11:44
  • The problem with this solution is that in the message converter I don't know what to do with the file yet. Save it to a file? Save to DB? etc. only the client side knows what to do with it. – shayma Nov 03 '14 at 12:44

2 Answers2

0

Check how Spring MVC handles large files upload with org.springframework.web.multipart.commons.CommonsMultipartResolver. It has a 'maxInMemorySize' that can help control the memory requirements. See this thread for using a multipart resolver with the REST template Sending Multipart File as POST parameters with RestTemplate requests

Community
  • 1
  • 1
Alexandru Ceausu
  • 317
  • 2
  • 9
0

As far as I am aware, RestTemplate's getForEntity() is not an appropriate way to get an InputStream. It's a convenience for converting to and from entity classes, so presumably that's where your problem lies.

Since you are used to HttpInputMessage, why don't you use HttpInputMessage.getBody() on the client side as well? It gets you a nice InputStream, which would be ready for passing straight to an OutputStream such as HttpServletResponse.getOutputStream().

mwarren
  • 2,409
  • 1
  • 22
  • 28
  • The converter itself doesn't know what to do with the file, only the code after the call knows. I can drop it into a temp file and return an FileInputStream of the file but that's inefficient. – shayma Nov 03 '14 at 13:50
  • Why do you save a file and then get an InputStream on the saved file, when you can go and get an InputStream on the url directly? I use HttpMethod.getResponseBodyAsStream() from Apache Commons, for example, to get an InputStream from a url. Surely you have some way of doing the same? Or have I misunderstood what you are trying to do? Showing a bit more code might be a good idea. – mwarren Nov 03 '14 at 15:00