1

I need to execute a series of remove file commands (listed below) to an Amazon S3 bucket from a pipe. But I need to execute them line by line using the Command Line Utility (v2) from Amazon S3 and I can't figure out how to do this. So SFTP has a batch file utility built in to read a series of remove commands from a text file but S3 doesn't have this ability. What's my best option here? (This has been taken from a > pipe to text file and I'm using terminals BSD on a mac.) 

aws s3 rm s3://xxx.yyy/FILE_25Min_PGM_05-11-2020.mp3
aws s3 rm s3://xxx.yyy/FILE_25Min_PGM_05-12-2020.mp3
aws s3 rm s3://xxx.yyy/FILE_25Min_PGM_05-13-2020.mp3
aws s3 rm s3://xxx.yyy/FILE_25Min_PGM_05-14-2020.mp3
aws s3 rm s3://xxx.yyy/FILE_25Min_PGM_05-15-2020.mp3
Martin Prikryl
  • 188,800
  • 56
  • 490
  • 992
nwood21
  • 312
  • 2
  • 13
  • what is your question? how to loop over a index.txt and parse file names into given pattern for execution? or how to create index.txt by given search criteria maybe? – alecxs Jun 03 '20 at 12:07
  • Quite difficult trying to ask the right question. I think my question is how to take the output of awk to run a series of commands. Above is the output of an awk command. So I think I need to use xargs or use the system command in awk as stated here:. https://stackoverflow.com/questions/20389799/using-output-of-awk-to-run-command – nwood21 Jun 03 '20 at 16:52
  • whats the input? is there a list or do you want to delete all FILE_25Min_PGM_05*.mp3? you can for example loop over a input.txt https://stackoverflow.com/a/37210472 – alecxs Jun 03 '20 at 17:10

1 Answers1

1

Here's the solution: | awk '{ system("aws s3 rm s3://xxx.yyy/" $4) }

$4 is the field which contains the filename. It's not super clear in the AWK manual that this will effectively run a "loop" not just outputting one command but one system command per row with the field specified. Very powerful if you have a list of files either in a text file or being piped into AWK that you want to run a system command on.

nwood21
  • 312
  • 2
  • 13