how to copy s3 object from one bucket to another using python boto3

Question

I want to copy a file from one s3 bucket to another. I get the following error:

s3.meta.client.copy(source,dest)
TypeError: copy() takes at least 4 arguments (3 given)

I'am unable to find a solution by reading the docs. Here is my code:

#!/usr/bin/env python
import boto3
s3 = boto3.resource('s3')
source= { 'Bucket' : 'bucketname1','Key':'objectname'}
dest ={ 'Bucket' : 'Bucketname2','Key':'backupfile'}
s3.meta.client.copy(source,dest)

This answer works https://stackoverflow.com/a/68849632/3177007 — Mohamed Jaleel Nazir, Aug 19 '21 at 14:22

score 127 · Accepted Answer · answered Nov 24 '17 at 07:32

127

You can try:

import boto3
s3 = boto3.resource('s3')
copy_source = {
      'Bucket': 'mybucket',
      'Key': 'mykey'
    }
bucket = s3.Bucket('otherbucket')
bucket.copy(copy_source, 'otherkey')

or

import boto3
s3 = boto3.resource('s3')
copy_source = {
    'Bucket': 'mybucket',
    'Key': 'mykey'
 }
s3.meta.client.copy(copy_source, 'otherbucket', 'otherkey')

Note the difference in the parameters

answered Nov 24 '17 at 07:32

Adarsh

3,273
3
20
44

4

How does this change when the source bucket is a public s3 bucket not under my account? – geominded May 19 '21 at 22:21
@geominded Check this out: https://aws.amazon.com/premiumsupport/knowledge-center/copy-s3-objects-account/ Hope it helps. – Adarsh Oct 11 '22 at 06:01
Is it just me or is this an extremely weird function? Source is a dictionary but destination is two strings? WTH AWS? And of course the key names are capitalized just to make absolute sure they don't abide by any convention... – foobarbecue Mar 14 '23 at 23:00
@foobarbecue Yup it is not the best way to go about it. Haven't checked the recent revision. Probably it will be fixed. If not can create an issue in their repo. – Adarsh Apr 18 '23 at 07:20

score 12 · Answer 2 · answered Nov 24 '17 at 07:28

12

Since you are using s3 service resource, why not use its own copy method all the way?

#!/usr/bin/env python
import boto3
s3 = boto3.resource('s3')
source= { 'Bucket' : 'bucketname1', 'Key': 'objectname'}
dest = s3.Bucket('Bucketname2')
dest.copy(source, 'backupfile')

answered Nov 24 '17 at 07:28

hjpotter92

78,589
36
144
183

`copy() missing 1 required positional argument: 'Key' ` – techkuz Feb 28 '20 at 11:39

score 9 · Answer 3 · edited Sep 14 '20 at 21:55

9

This is the syntax from docs:

import boto3

s3 = boto3.resource('s3')
copy_source = {
    'Bucket': 'mybucket',
    'Key': 'mykey'
}
s3.meta.client.copy(copy_source, 'otherbucket', 'otherkey')

edited Sep 14 '20 at 21:55

Asclepius

57,944
17
167
143

answered Nov 24 '17 at 07:27

Jibran

873
7
12

how to copy s3 object from one bucket to another using python boto3

3 Answers3

Linked