How can you read select few files from a tar file in s3 without having to download the tar?

Question

You do not want to download the tar file. Just download select few files inside the tar.

Does s3 provide any API to do this or is Apache Commons Compress my best bet?

I played around with doing this in Python here: https://stackoverflow.com/questions/56086604/how-to-list-files-inside-tar-in-aws-s3-without-downloading-it — parsley72, May 11 '19 at 05:50

score 2 · Answer 1 · answered Jul 27 '17 at 15:01

While you'll be able to stream the file from S3 you'll still basically download it. The Apache Commons Compress library will help hide some of this and is a good solution. The other option would be to store the individual files in the tar file so that they can be randomly accessed.

How can you read select few files from a tar file in s3 without having to download the tar?

1 Answers1