-1

I am using this command but unable to rename and move a file from one s3 folder to another

I want to rename this file as well as move

Here anagaza.csv000 is the file I want to rename and move it to movedData folder with the name angaza.csv

Once I move the file I also want a more generic approach where instead of file name like anagaza.csv000 I could use wildcard like anaga*.* or "wildcard.csv*" something like that but my destination should have the name I choose to give which is angaza.csv

aws s3 --recursive mv 
s3://mygluecrawlerbucket/angaza_accounts/to_be_processed/anagaza.csv000   
s3://mygluecrawlerbucket/angaza_accounts/movedData/angaza.csv --profile default
Paul Hodges
  • 13,382
  • 1
  • 17
  • 36
bigDataArtist
  • 141
  • 1
  • 12
  • What is the exact logic you would want to use when renaming/moving the object? That is, how does it go from `anagaza.csv000` to `angaza.csv`? Will it always just be a matter of removing `000` from the end, or will there be different naming formats? Also, why have you included `--recursive`? – John Rotenstein Aug 01 '21 at 23:24

1 Answers1

1

You can generate new object names with shell script (mv or rename) and after that rename in S3 with s3 mv one by one. For renaming the name of objects, you need to write regex or somethig by yourself. For renaming with regex, this SO will help.

Here is sample.sh I used.

list_objects=$(aws s3 ls s3://oldbucket | awk '{print $4}')

for old_object_name in $list_objects; do
    new_object_name=$(...) # mv or rename
    aws s3 mv s3://oldbucket/$old_object_name s3://newbucket/$new_object_name
done
shimo
  • 2,156
  • 4
  • 17
  • 21