0

In my app, users are able to upload images to their folder in an S3 bucket. Current requests in relation to uploading and retrieving images from S3 are handled through API Gateway, users are authenticated with Cognito.

Is there a way where I could list all objects in a folder of a bucket and be able to display all of the images?

I'm asking the same question as this one, except I only want authenticated users to be able to access it.

Is there a way I could retrieve them through a Lambda function?

Any help appreciated.

Update

I have tried to implement S3's listObjectsV2 function which does not return resigned URLs with each object. Is there a way to achieve this?

Response from function:

{
    "IsTruncated": false,
    "Contents": [
        {
            "Key": "username/image/Screenshot 2021-02-22 at 13.24.55.png",
            "LastModified": "2021-03-07T19:43:05.000Z",
            "ETag",
            "Size": 22436,
            "StorageClass": "STANDARD"
        }
    ],
    "Name": "user-images",
    "Prefix": "username/",
    "MaxKeys": 1000,
    "CommonPrefixes": [],
    "KeyCount": 1
}
Alex Goodison
  • 41
  • 6
  • 1
    You must have already solved authentication and authorization if your users are able to securely upload to only their personal folder, no? – jarmod Mar 07 '21 at 20:08
  • 1
    Yes, that is trivially possible, depends on the language: https://stackoverflow.com/questions/42394429/aws-sdk-s3-best-way-to-list-all-keys-with-listobjectsv2 https://alexwlchan.net/2017/07/listing-s3-keys/ etc. there a thousands of questions regarding this. – luk2302 Mar 07 '21 at 20:08
  • @jarmod Yes but I want to be able to call a function on the backend like `listObjects` that can return links to view images. – Alex Goodison Mar 07 '21 at 22:49
  • @luk2302 thanks for the response, in my case, listing keys isn’t sufficient to be able to show a preview of each photo because each photo is protected with Cognito Auth... or else I’m being blind here but that’s what I understand of it – Alex Goodison Mar 07 '21 at 22:52
  • 1
    Typically the S3 objects are not public and your back-end gets the list of objects and then generates temporary, pre-signed URLs for each so that the front-end can retrieve them. – jarmod Mar 07 '21 at 23:07
  • @jarmod I tried to use the S3 `listObjectsV2` function and it didn't return any pre-signed URL for any of the objects. (Please see the original post which I have updated with the response I got from the function). Is there a way to retrieve a list of objects with pre-signed URLs. – Alex Goodison Mar 16 '21 at 20:48
  • The `listObjectsV2` SDK doesn't return pre-signed URLs. That's not its purpose. It lists objects. You then decide which objects you need pre-signed URLs for and then call the relevant SDK function to get a pre-signed URL for each. – jarmod Mar 16 '21 at 20:55

1 Answers1

1

The recommended method to serve private content is:

  • Do not use a Bucket Policy
  • Keep all objects as private
  • Users authenticate to your application
  • When a user requests access to a private object, or you wish to include a private object on a page (eg in <img src="..."> tags), the application should check whether the user is permitted to access the object
  • If they are permitted to access the object, the application generates an Amazon S3 pre-signed URLs, which provides time-limited access to a private object
  • When the user's browser sends the pre-signed URL to S3, Amazon S3 will check that the URL is correct and that the time period has not expired. If it is okay, it will return back the private object

This way, your application manages all permissions related to access, but the content itself is served from Amazon S3. This allows complex access logic, such as users sharing photos with other users.

John Rotenstein
  • 241,921
  • 22
  • 380
  • 470
  • Hi John, thanks for the response. I'm looking for a way to retrieve a list of images under the same folder in an S3 bucket in one function call. It would need to return pre-signed URLs for the images too, as objects are private. Is there a function that allows this? Thanks. – Alex Goodison Mar 16 '21 at 20:50
  • @AlexGoodison yes, there's a way to do this with one function. *You* write one function and have it do the multiple, necessary things (list objects, filter as needed, create pre-signed URLs). – jarmod Mar 16 '21 at 21:02
  • @AlexGoodison Pre-signed URLs can be calculated in your own code, without the need to make an API call. It's basically a hash function using the Secret Key. – John Rotenstein Mar 16 '21 at 21:17