boto3 how to create object with metadata?

Question

In the example below I want to set a timestamp metadata attribute when created an S3 object. How do I do that? The documentation is not clear.

import uuuid
import json
import boto3
import botocore
import time

from boto3.session import Session
session = Session(aws_access_key_id='XXX',
                  aws_secret_access_key='XXX')

s3 = session.resource('s3')

bucket = s3.Bucket('blah')

for filename in glob.glob('json/*.json'):
    with open(filename, 'rb') as f:
        data = f.read().decode('utf-8')
        timestamp = str(round(time.time(),10))
        my_s3obj = s3.Object('blah', str(uuid.uuid4())).put(Body=json.dumps(data))

score 18 · Answer 1 · edited Apr 19 '23 at 00:41

18

As for boto3, You have the upload_file() option detailed in the boto3 website here.

import boto3
s3 = boto3.client('s3')
s3.upload_file('/tmp/hello.txt', 'mybucket', 'hello.txt')

While uploading a file, you have to specify the key (which is basically your object/file name). Adding the metadata when creating the key would be done using the "ExtraArgs" option:

s3ressource.upload_file(
    Filename, bucketname, key, 
    ExtraArgs={
        "Metadata": {
            "metadata1": "ImageName",
            "metadata2": "ImagePROPERTIES",
            "metadata3": "ImageCREATIONDATE"
        }
    }
)

edited Apr 19 '23 at 00:41

Leaderboard

371
1
10

answered Apr 24 '17 at 14:24

MouIdri

1,300
1
18
37

6

Great answer- one minor addition: If you want to modify one of the defined Object metadata keys (Content-Encoding, Content-Type, etc.) you must use them directly in `ExtraArgs` (i.e. `ExtraArgs={"ContentEncoding": "gzip"}`) – Devin Cairns Jan 25 '19 at 21:26
Yes, it depends on the server... AWS has 2KB while onpremises system can have more. http://docs.netapp.com/sgws-112/topic/com.netapp.doc.sg-s3/GUID-0BEF8D16-B743-4BF6-8E27-A5FE4D33C1F0.html – MouIdri Mar 12 '21 at 14:41
@Tommy, Surround your code with 3 backticks python CODE 3 backticks No space between the top 3 backticks and the language name like python in this case. – Montaro Sep 01 '22 at 11:04

David Morales · Answer 2 · 2015-11-18T05:20:03.243

9

You can specify metadata for the object as key-value pairs like this:

s3.Object('bucket-name', 'uuid-key-name').put(Body='data', 
                                              Metadata={'key-name':'value'})

See the boto3 docs for other parameters you can use inside put().

edited Nov 18 '15 at 05:20

answered Nov 18 '15 at 05:13

David Morales

870
7
14

boto3 how to create object with metadata?

2 Answers2

Linked