0

The method below in my Java SpringBoot application directly streams and uploads a InputStream to an Amazon S3 bucket using the TransferManager, and the InputStream is derived by the MultipartFile. I found that the code costs so many memory and I don't know why. I also found some similar questions: High memory usage when uploading a multipart file to Amazon S3 via streaming? and Java Heap Space is insufficient to upload files on AWS S3, but nothing help. Can anyone give me some help or suggestions, thank you.

public void upload(String key, MultipartFile file) throws AmazonClientException, InterruptedException, IOException {
        TransferManager tm = null;
        try {
            ObjectMetadata objectMetadata = new ObjectMetadata();
            objectMetadata.setContentLength(file.getSize());
            tm = TransferManagerBuilder.standard().withS3Client(s3).withMinimumUploadPartSize(5 * 1024 * 1024L).withMultipartUploadThreshold(5 * 1024 * 1024L).build();
            Upload upload = tm.upload(s3Config.getBucket(), key, file.getInputStream(), objectMetadata);
            upload.waitForCompletion();
        } finally {
            input.close();
            if (tm != null) {
                tm.shutdownNow();
            }
        }
    }
Gavin
  • 47
  • 5
  • Have tried filing an issue on [github](https://github.com/aws/aws-sdk-java)? – Turing85 Aug 08 '20 at 08:39
  • @Turing85 Not yet, I don't know whether it is a issue – Gavin Aug 08 '20 at 08:43
  • There is no harm in asking. After all, the aws-sdk developers are more likely to give you a more qualified answer than we can. – Turing85 Aug 08 '20 at 08:44
  • 1
    You are right, I'll go to create an issue – Gavin Aug 08 '20 at 08:53
  • Not sure whether your code snippet is accurate, but it seems you're opening a stream without closing it. (see `file.getInputStream()`). – blagerweij Oct 08 '20 at 13:42
  • Please also take a look here https://stackoverflow.com/a/64263423/1704634 , it uses an S3OutputStream which automatically switches to multipart uploads in case the stream is too large. Currently uses a 10MB buffer, but this can be configured smaller/larger. – blagerweij Oct 08 '20 at 15:40

0 Answers0