Creating presigned url for a S3 folder in python

Question

I'm trying to generate a presigned url to a S3 folder (which itself contains more folders/files) and distribute it between my clients so they can download its content. i.e. by clicking the link, the users will download the folder to their local disk.

However, I keep getting a "no such key" error in an XML dialogue.

I'm using client.generate_presigned_url() from boto3 sdk

def create_presigned_url(bucket, object):
    try:
        url = s3_client.generate_presigned_url(
            'get_object',
            Params={
                'Bucket': bucket,
                'Key': object
            },
            ExpiresIn=240,
            HttpMethod='GET'
        )
    except ClientError as e:
        print(e)
        return None
    return url

this is the error message:

This XML file does not appear to have any style information associated with it. The document tree is shown below.
<Error>
   <Code>NoSuchKey</Code>
      <Message>The specified key does not exist.</Message>
         <Key>output/BARNES/070419/APR19BAR/</Key>
         <RequestId>E6BE736FE945FA22</RequestId>
         <HostId>
      hk3+d+***********************************************************+EO2CZmo=
          </HostId>
</Error>

Your `object` variable has to be a key to a single S3 object. As in the answer below a "folder" is not really an object in S3, its a mere part of a key to your actual file. — Tommy, Jun 25 '19 at 10:39

score 7 · Accepted Answer · answered Jun 25 '19 at 10:29

7

S3 has no concept of "folders". What you are effectively trying to do here is create a presigned url for multiple keys which is also not possible. If you absolutely have to share single url for multiple files, you'll need zip them into a single object and then share key of that object using presigned url.

answered Jun 25 '19 at 10:29

Ninad Gaikwad

4,272
2
13
23

Thanks Ninad, That option already crossed my mine. Although it sounds promising, I don't want to hold two copies of the same data. – Yotam Raz Jun 25 '19 at 10:43
2

A presigned URL is only applicable to a single object. So you'll have to try something else if you want users to be able to download multiple objects. – Ninad Gaikwad Jun 25 '19 at 10:47

score 4 · Answer 2 · answered Apr 30 '20 at 13:27

import boto3

s3_client = boto3.client(
    's3',
    region_name='your_region_name',
    aws_access_key_id='your_aws_access_key_id',
    aws_secret_access_key='your_aws_access_key_id',
)

# Just specify folder_name:
url = s3_client.generate_presigned_url(
    ClientMethod='put_object',
    Params={'Bucket': 'your_bucket_name', 'Key': 'folder_name/file_name.txt',},
    ExpiresIn=60,
)

score 0 · Answer 3 · edited Jul 15 '20 at 08:21

The generated URL will expire after a week.

Since there is a restriction by AWS: you can keep a link at most one week.
That is approximately 600000 seconds (See ExpiresIn).

def create_presigned_url(bucket, object):
    try:
        bucket_name = bucket
        ACCESS_KEY = "access_key"
        SECRET_KEY = "secret_key"
        key = object
        location = boto3.client('s3', aws_access_key_id=ACCESS_KEY,aws_secret_access_key=SECRET_KEY).get_bucket_location(Bucket=bucket_name)['LocationConstraint']
        s3_client = boto3.client(
                's3',
                region_name=location,
                aws_access_key_id=ACCESS_KEY,
                aws_secret_access_key=SECRET_KEY,
            )
        url = s3_client.generate_presigned_url(
                ClientMethod='get_object',
                Params={'Bucket': bucket_name, 'Key': key, },
                ExpiresIn=600000,
            )
    except ClientError as e:
        print(e)
        return None
    return url

Creating presigned url for a S3 folder in python

3 Answers3