1

I need to copy the [EDIT: entire] contents of a bucket into another bucket. The answers to this question implies that it can be done on the CLI, [EDIT: as a single operation (rather than looping over every record in the bucket)]

Can it be done via any C# SDKs? We're currently using the AWSSDK.S3 Nuget package SDK, but I can't see any methods in that package's AmazonClient that would allow this sort of copy?

smac2020
  • 9,637
  • 4
  • 24
  • 38
Brondahl
  • 7,402
  • 5
  • 45
  • 74

2 Answers2

2

The AWS SDK for .NET supports copying objects from one bucket to another. However, you cannot copy ALL objects in a single call. See REF docs here:

https://docs.aws.amazon.com/sdkfornet/v3/apidocs/index.html?page=TS3Client.html&tocid=Amazon_S3_AmazonS3Client

The business logic to copy an object from one bucket to another can be found in the AWS Code Library, which contains hundreds of tested examples in supported programming languages.

For this use case, see:

enter image description here

URL - https://docs.aws.amazon.com/code-library/latest/ug/s3_example_s3_CopyObject_section.html

The code lib should be the first place to reference when you want to learn how to use the AWS SDK.

smac2020
  • 9,637
  • 4
  • 24
  • 38
  • Yes, I know that I can copy a single object. But I'm looking to copy the entire bucket; and I'd rather not do it object-by-object if possible. But the code library doesn't have any links to anything that describe that :( – Brondahl Jan 18 '23 at 15:53
  • Question edited to clarify goal. – Brondahl Jan 18 '23 at 15:55
  • You are correct - looking at the .NET REF docs -- there is not a method that copies all objects at once. You would need to do it the way shown in the example above. See V3 client here -- https://docs.aws.amazon.com/sdkfornet/v3/apidocs/items/S3/TS3Client.html – smac2020 Jan 18 '23 at 16:03
0

S3 Batch Operations, AWS Datasync, or S3 Replication could be options as well. These tools require more initial setup, but the actual data transfer will happen in a single request. If there are a lot of objects to copy or this process is run frequently, it may be easier to automate using one of these options.

See https://aws.amazon.com/blogs/storage/considering-four-different-replication-options-for-data-in-amazon-s3/ for a comparison of a few options.

arjunrawal
  • 101
  • 3