how to copy s3 object from one bucket to another using python s3fs

Question

Using python s3fs, how do you copy an object from one s3 bucket to another? I have found answers using boto3, but could not find anything when looking through the s3fs docs.

Is there a reason why this should be done with boto3 instead of s3fs? — jjbskir, Jul 12 '23 at 20:58

score 0 · Answer 1 · answered Jul 12 '23 at 18:41

Open from one bucket and write to another:

s3 = S3FileSystem(...)

# You'll want to set a block size
# if your files are particularly large
block_size = 2**20

with s3.open('bucket_a/file', 'rb', block_size=block_size) as infile, s3.open('bucket_b/file', 'wb') as outfile:
    for line in infile:
        outfile.write(line)

how to copy s3 object from one bucket to another using python s3fs

1 Answers1