0

I'm new to Java, and very new to Spring, Rest, and IO Streams, so apologies.

What I need to do is construct a zip file using ZipOutputStream, and then send that file to a REST endpoint as multipart file data.

I've been told that it should be possible to do this without holding the entire zip file in memory (it could be very large), by doing some clever streaming.

Something like creating a stream, then passing it to the rest template, and then writing the zip to the stream x bytes at a time, and the rest template will automatically stream each new block of bytes over the HTTP request as they get written without having to assemble the entire file on disk or in memory first (or maybe the stream has some kind of callback lambda that is invoked by the rest template trying to read the stream, so I can append more data to it)

Is that possible?

I've found similar posts, but none that help me understand how to do this, if it's at all possible. POST InputStream RestTemplate How to forward large files with RestTemplate?

Vectorspace
  • 1
  • Yes, that is absolutely possible. How to implement it depends on the framework you are using for your HTTP endpoints. – Jorn Jul 28 '23 at 11:49
  • @Jorn - thanks for replying. It's not our own endpoint we're calling. We're going to be calling the external endpoint using RestTemplate. Our application uses Spring Boot. Is that what you needed to know? – Vectorspace Jul 28 '23 at 12:06
  • Same difference, but now it depends on the HTTP client you are using. For Spring's `RestTemplate`, I'm not sure but you should google for "spring resttemplate post write to stream" or something. – Jorn Jul 28 '23 at 12:09
  • I did that search, it's how I found the linked posts. But the bit I can't find is how to have Rest send each chunk from the stream as its written to it, as opposed to loading all the bytes into the stream before REST sends it. – Vectorspace Jul 28 '23 at 12:22
  • You don't need to do anything special. There might be some in-between bufffers, but writing to the stream will already send chunks when possible. – Jorn Jul 28 '23 at 12:27
  • I still can't see how to set it up except by having RestTemplate send a byte stream that is already completely filled, which means I have the entire file in memory which defeats the object. What also occurs to me is that I don't know the final size of the zip file until it is built, but surely RestTemplate would need to know that up front? The Resource class that I've seen examples use to encapsulate what is being sent has a getter method for the total length fo the data. – Vectorspace Jul 28 '23 at 13:01
  • Out of interest, do you mean a zip *archive* containing n entries or is it one file that's been zipped? – g00se Jul 28 '23 at 13:17
  • HTTP requests have data that isn't fully known up front all the time. That size parameter isn't mandatory. As for sending the data, you don't "send a filled byte stream", you *write* to the *existing* byte stream. So don't make your own stream, write to the one provided by `RestTemplate`. – Jorn Jul 28 '23 at 13:22
  • Thanks for your patience, I'm sorry that I'm not understanding. All the examples I find look like this: `response=restTemplate.doSomething(url,content)` A call in which you pass in an object containing all the data (or an input stream e.g. a FileInputStream), and get back the response from processing the http request. I assume I would need to do something where I "start" the restTemplate, then write to some stream from somewhere, and then tell restTemplate that I'm done and it can complete the http request and return the response. I can find nothing that looks like that – Vectorspace Jul 28 '23 at 14:10
  • @goose I have a list of file paths referring to one or more files on disk. I need to create a zip file containing all those files, and then send them as multipart file data to a REST endpoint. I'm looking for a way fo doing that without having to A. write the entire zip file to disk and then read it back, or B. having the entire zip file in memory – Vectorspace Jul 28 '23 at 14:12

0 Answers0