16

I am attempting to move a file, from on s3 location to another, using an activity in a AWS data pipeline.

The command I am using is:

(aws s3 mv s3://foobar/Tagger/out//*/lastImage.txt s3://foobar/Tagger/testInput/lastImage.txt)

But I receive the following error:

A client error (404) occurred when calling the HeadObject operation: Key "Tagger/out//*/lastImage.txt" does not exist

But, if I replace the "*" with the specific directory name, it will work. The problem is I won't always know the name of the directory, so I was hoping I could use the "*" as a wild card.

Tom
  • 2,689
  • 16
  • 21
BrainPermafrost
  • 644
  • 2
  • 7
  • 20
  • 1
    AWS SDK does not support wildcards. Instead, you can search based on prefix (stuff before the wildcard). Then, you could grep the results for those ending in `/lastImage.txt` – Matt Houser Jul 25 '15 at 18:26

1 Answers1

24

Wildcards in the AWS S3 CLI only work when using the --recursive flag.

So this should work for you:

aws s3 mv s3://foobar/Tagger/out/ s3://foobar/Tagger/testInput/ --recursive --exclude "*" --include "*/lastImage.txt"

Unfortunately, this will recreate the entire directory structure in your target location, and I'm not immediately sure that can be solved by just using the AWS CLI.

Vao Tsun
  • 47,234
  • 13
  • 100
  • 132
shrikant
  • 951
  • 6
  • 9