Stream-uploading large files via Spring WebClient

Question

I am having the same problem as referred to here.

Stream upload 'POST' in Spring WebClient

I need to upload large files via Spring WebClient and I don't want to read the files in memory to do that.

I have the files stored in Minio S3 bucket and the interface allows me to receive the file as java.io.InputStream.

Then I would like to stream that as an upload body to another service endpoint.

The solution, which gives an error of Content type 'application/octet-stream' not supported for bodyType=org.springframework.web.reactive.function.BodyInserters is produced the following way:

public String uploadFile(String fileId, String fileName) {
        InputStream fileInputStream = minioClient.getObject(bucketName, fileId);

        return webClient.post()
                .uri(properties.getUrl())
                .contentType(APPLICATION_OCTET_STREAM)
                .headers(httpHeaders -> {
                    httpHeaders.set(FILE_NAME, fileName);
                })
                .bodyValue(BodyInserters.fromResource(fileInputStream)
                .retrieve()
                .bodyToFlux(String.class)
                .blockFirst();
    }

score 1 · Answer 1 · answered Nov 25 '20 at 14:57

You can try using DataBufferUtils.readInputStream(..) (javadoc) to create a Flux<DataBuffer>. And then use the appropriate BodyInserters.fromDataBuffers(..) (javadoc) to pass to .bodyValue(). This should correctly not load the entire file into memory, but rather buffer and send it chunk by chunk.

I'm not sure if this will work with application/octet-stream as the content type as I haven't been able to test it.

score 0 · Answer 2 · answered May 12 '22 at 21:43

Replace .bodyValue() with .body() and it will work.

     public String uploadFile(String fileId, String fileName) {
        InputStream fileInputStream = minioClient.getObject(bucketName, fileId);

        return webClient.post()
                .uri(properties.getUrl())
                .contentType(APPLICATION_OCTET_STREAM)
                .headers(httpHeaders -> {
                    httpHeaders.set(FILE_NAME, fileName);
                })
                .body(BodyInserters.fromResource(fileInputStream)
                .retrieve()
                .bodyToFlux(String.class)
                .blockFirst();
    }

Stream-uploading large files via Spring WebClient

2 Answers2

Linked