How to download s3 object directly into memory in java

Question

Is it possible to download S3object in Java directly into memory and get it removed when i'm done with the task?

score 58 · Answer 1 · edited Jan 08 '19 at 12:08

58

Use the AWS SDK for Java and Apache Commons IO as such:

//import org.apache.commons.io.IOUtils

AmazonS3 s3  = new AmazonS3Client(credentials);  // anonymous credentials are possible if this isn't your bucket
S3Object object = s3.getObject("bucket", "key"); 
byte[] byteArray = IOUtils.toByteArray(object.getObjectContent());

Not sure what you mean by "get it removed", but IOUtils will close the object's input stream when it's done converting it to a byte array. If you mean you want to delete the object from s3, that's as easy as:

s3.deleteObject("bucket", "key");

edited Jan 08 '19 at 12:08

Xavier Guihot

54,987
21
291
190

answered Nov 01 '12 at 19:52

Zach Musgrave

1,012
7
7

Thanks for your help!this would require the use of additional libraries. Wouldn't it be better if i loop through input stream to get the byte array? – Manisha Nov 03 '12 at 14:50
1

@Manisha, see http://stackoverflow.com/questions/1264709/convert-inputstream-to-byte-array-in-java. – Paul Draper Jul 10 '15 at 14:51
9

You can now do this without `org.apache.commons.io`. The AWS SDK ships this method in it's own IOUtils: `import com.amazonaws.util.IOUtils;` – nbarraille Feb 22 '19 at 07:28

score 6 · Answer 2 · answered May 21 '20 at 16:53

As of AWS JAVA SDK 2 you can you use ReponseTransformer to convert the response to different types. (See javadoc).

Below is the example for getting the object as bytes

GetObjectRequest request = GetObjectRequest.builder().bucket(bucket).key(key).build()
ResponseBytes<GetObjectResponse> result = bytess3Client.getObject(request, ResponseTransformer.toBytes())

// to get the bytes
result.asByteArray()

score 1 · Answer 3 · edited Feb 17 '20 at 08:31

1

For example, convert file data to string:

S3Object s3object = s3.getObject(new GetObjectRequest(bucketName, key));
S3ObjectInputStream inputStream = s3object.getObjectContent();
StreamUtils.copyToString(inputStream, StandardCharsets.UTF_8);

edited Feb 17 '20 at 08:31

סטנלי גרונן

2,917
23
46
68

answered Feb 17 '20 at 08:05

benderalex5

129
1
3

score 1 · Answer 4 · answered Apr 21 '21 at 23:42

1

This is the best practise.

S3Object object = s3.getObject("bucket", "key"); 
byte[] byteArray = IOUtils.toByteArray(object.getObjectContent());

answered Apr 21 '21 at 23:42

Cemalettin Altınsoy

137
2
6

How to download s3 object directly into memory in java

4 Answers4