0

We are building java based GUI to upload very large files to S3 bucket. We get pre-signed URL from API and we hit this URL to upload our file.

Now since file size is very large (10+ GB), we want to show user data transfer stats (e.g. how much data was transferred, transfer rate etc).

Important Note - We dont get AWS ID-KEY to access bucket so we can't use AWS SDK.

Any pointer will be helpful and appreciated.

Christopher Moore
  • 15,626
  • 10
  • 42
  • 52
navaltiger
  • 884
  • 12
  • 27
  • Based on your restrictions, this type of s3 upload should not be any different from regular file upload progress bar. You can check for some answer here https://stackoverflow.com/questions/254719/file-upload-with-java-with-progress-bar – qkhanhpro Apr 17 '20 at 07:26

1 Answers1

1

I'm going to start by strongly recommending that you rethink your decision to PUT signed URLs. With a single PUT, any network hiccup means that you'll have to start from the beginning, which will be painful for anyone using anything slower than a Gbit network. The Java SDK provides a transfer manager that will automatically break the file into chunks, and resend any chunks that fail. Plus it gives you a progress reporter.

YES, I know that you said that you don't get API keys. But whatever is giving you a signed URL could just as easily provide you with API keys that are scoped to a single file upload.

OK, you're not going down that route. Unfortunately, you don't show the code that you're currently using to do the upload, so we have no idea what options you have available. So I'll assume that you're using the Apache Commons HttpClient, and point you at this Stack Overflow answer.

parsifal
  • 71
  • 1