10

I have the s3cmd command line tool for linux installed. It works fine to put files in a bucket. However, I want to move a file into a 'folder'. I know that folders aren't natively supported by S3, but my Cyberduck GUI tool converts them nicely for me to view my backups.

For instance, I have a file in the root of the bucket, called 'test.mov' that I want to move to the 'idea' folder. I am trying this:

s3cmd mv s3://mybucket/test.mov s3://mybucket/idea/test.mov

but I get strange errors like:

WARNING: Retrying failed request: /idea/test.mov (timed out)
WARNING: Waiting 3 sec...

I also tried quotes, but that didn't help either:

s3cmd mv 's3://mybucket/test.mov' 's3://mybucket/idea/test.mov'

Neither did just the folder name

s3cmd mv 's3://mybucket/test.mov' 's3://mybucket/idea/'

Is there a way within having to delete and reput this 3GB file?

Update: Just FYI, I can put new files directly into a folder like this:

s3cmd put test2.mov s3://mybucket/idea/test2.mov

But still don't know how to move them around....

Rodrigue
  • 3,617
  • 2
  • 37
  • 49
Nic Cottrell
  • 9,401
  • 7
  • 53
  • 76

3 Answers3

14

To move/copy from one bucket to another or the same bucket I use s3cmd tool and works fine. For instance:

s3cmd cp --r s3://bucket1/directory1 s3://bucket2/directory1
s3cmd mv --recursive s3://bucket1/directory1 s3://bucket2/directory1
sgimeno
  • 1,883
  • 1
  • 22
  • 35
  • 1
    the original poster is NOT asking how to copy between S3 buckets, but how to rename a file within an S3 bucket! – Tilo Aug 02 '13 at 01:15
  • 3
    Isn't renaming a file the same as moving it? At least in Unix systems I always rename my files using mv command, so my point was that if s3cmd mv isn't working for files you can workaround this moving the whole directory... – sgimeno Aug 02 '13 at 12:32
  • The difference is that the files are very large and I want to structure the files without downloading and reuploading them... – Nic Cottrell Aug 14 '13 at 07:25
  • 1
    This is an extremely old post, but I wanted to chime in that S3 is an immutable object store. You can't rename an object without "moving" it. S3 offers primitives that allow you to do this (COPY/DELETE) without needing to download and re-upload objects. – Brandon Galbraith Nov 17 '15 at 19:58
  • And if you are wondering if `s3cmd mv --recursive` is the simple atomic operation you are used to on a Unix filesystem, be warned that, as Brandon implies, it is **not** and if you have a lot of data in the hierarchy you might want to run this in screen or tmux. – dannyman Apr 06 '16 at 22:28
0

Probably your file is quite big, try increasing socket_timeout s3cmd configuration setting

http://sumanrs.wordpress.com/2013/03/19/s3cmd-timeout-problems-moving-large-files-on-s3-250mb/

pastk
  • 41
  • 3
  • But shouldn't I be able to "move" a file already on S3 without having to download and then re-upload it? like in FTP... – Nic Cottrell Aug 14 '13 at 07:24
-2

Remove the ' signs. Your code should be:

s3cmd mv s3://mybucket/test.mov s3://mybucket/idea/test.mov

Also try what are the permissions of your bucket - for your username you should have all the permissions.

Also try to connect CloudFront to your bucket. I know it doesn' make sense but I have similar problem to bucket which do not have cloudfront instance clonnected to it.

ispasov
  • 51
  • 1
  • 11