Stack Overflow
Stack Overflow

Questions tagged [s3cmd]

S3CMD QUESTIONS MUST BE PROGRAMMING RELATED. s3cmd is a command line tool for uploading, retrieving and managing data in Amazon S3. It is best suited for power users who don't fear command line. It is also ideal for scripts, automated backups triggered from cron, etc.

S3CMD QUESTIONS MUST BE PROGRAMMING RELATED. s3cmd is a command line tool for uploading, retrieving and managing data in Amazon S3. It is best suited for power users who don't fear command line. It is also ideal for scripts, automated backups triggered from cron, etc.

See the s3cmd homepage for more information.

282 questions
123
votes
28 answers

How do I delete a versioned bucket in AWS S3 using the CLI?

I have tried both s3cmd: $ s3cmd -r -f -v del s3://my-versioned-bucket/ And the AWS CLI: $ aws s3 rm s3://my-versioned-bucket/ --recursive But both of these commands simply add DELETE markers to S3. The command for removing a bucket also doesn't…
NobleUplift
  • 5,631
  • 8
  • 45
  • 87
104
votes
22 answers

What is the algorithm to compute the Amazon-S3 Etag for a file larger than 5GB?

Files uploaded to Amazon S3 that are smaller than 5GB have an ETag that is simply the MD5 hash of the file, which makes it easy to check if your local files are the same as what you put on S3. But if your file is larger than 5GB, then Amazon…
broc.seib
  • 21,643
  • 8
  • 63
  • 62
97
votes
10 answers

how to add cache control in AWS S3?

I have moved 20000 files to AWS S3 by s3cmd command. Now i want to add cache-control for all images (.jpg) These files are located in ( s3://bucket-name/images/ ). How can i add cache-control for all images by s3cmd or is there any other way to add…
Rajaraman
  • 1,637
  • 5
  • 19
  • 24
92
votes
4 answers

Exclude multiple folders using AWS S3 sync

How to exclude multiple folders while using aws s3 syn ? I tried : # aws s3 sync s3://inksedge-app-file-storage-bucket-prod-env \ s3://inksedge-app-file-storage-bucket-test-env \ --exclude 'reportTemplate/*…
Ashish Karpe
  • 5,087
  • 7
  • 41
  • 66
77
votes
6 answers

How to delete or purge old files on S3?

Are there existing solutions to delete any files older than x days?
Erik
  • 4,268
  • 5
  • 33
  • 49
53
votes
1 answer

Difference between s3cmd, boto and AWS CLI

I am thinking about redeploying my static website to Amazon S3. I need to automate the deployment so I was looking for an API for such tasks. I'm a bit confused over the different options. Question: What is the difference between s3cmd, the Python…
user1781186
46
votes
12 answers

How can I download a file from an S3 bucket with wget?

I can push some content to an S3 bucket with my credentials through S3cmd tool with s3cmd put contentfile S3://test_bucket/test_file I am required to download the content from this bucket in other computers that don't have s3cmd installed on them,…
David
  • 1,469
  • 5
  • 33
  • 51
35
votes
6 answers

Why does file uploaded to S3 have content type application/octet-stream unless I name the file .html?

Even though I set content type to text/html it ends up as application/octet-stream on S3. ByteArrayInputStream contentsAsStream = new ByteArrayInputStream(contentAsBytes); ObjectMetadata md = new…
Paul Taylor
  • 13,411
  • 42
  • 184
  • 351
31
votes
3 answers

s3cmd get an entire directory

I have a directory in my s3 that has only another directory inside it, but I don't know it's name. Using command line "get" tools, i'd like to download that directory, but it doesn't seems to want to. The structure looks…
hogsolo
  • 13,565
  • 6
  • 24
  • 25
31
votes
5 answers

S3 moving files between buckets on different accounts?

I'm doing some work for a client that has 2 separate AWS accounts. We need to move all the files in a bucket on one of their S3 accounts to a new bucket on the 2nd account. We thought that s3cmd would allow this, using the format: s3cmd cp…
Geuis
  • 41,122
  • 56
  • 157
  • 219
28
votes
3 answers

Creating a folder via s3cmd (Amazon S3)

I am using s3cmd to upload files to my S3 server. My problem is that when a directory on the server does not exist the upload fails. How can I tell s3cmd to create the folder if it does not exist? I am using PHP.
John
  • 301
  • 1
  • 3
  • 4
27
votes
4 answers

Zip an entire directory on S3

If I have a directory with ~5000 small files on S3, is there a way to easily zip up the entire directory and leave the resulting zip file on S3? I need to do this without having to manually access each file myself. Thanks!
Jin
  • 6,055
  • 2
  • 39
  • 72
22
votes
2 answers

Downloading pattern matched entries from S3 bucket

I have a S3 bucket in which there are several log files stored having the format index.log.yyyy-mm-dd-01 index.log.yyyy-mm-dd-02 . . . yyyy for year, mm for month and dd for date. Now i want to download only a few of them. I saw…
Shrikant Kakani
  • 1,511
  • 2
  • 17
  • 37
17
votes
4 answers

Necessary s3cmd S3 permissions for PUT/Sync

In moving to AWS EC2, I want to restrict my instances' user permissions for good reason. One thing the instances need to do is access files on S3 and write files there. However, I cannot find any way to achieve this without giving all permissions to…
Joseph Lust
  • 19,340
  • 7
  • 85
  • 83
15
votes
1 answer

Trying to set metadata on Amazon S3 file using s3cmd

I have a file on amazon S3 that has the suffix ".bin.gz". I want web browsers to treat it as a gzipped HTML file. So, I am able to go into the Amazon S3 web console (https://console.aws.amazon.com/s3/home) and navigate to the file and select it.…
chaimp
  • 16,897
  • 16
  • 53
  • 86
1
2 3
…
18 19