1

I am trying to create a Bash Script that can delete all of my Amazon Transcribe jobs from a single command. I have created the following script:

#!/bin/bash

aws transcribe list-transcription-jobs --max-results 100 --query '[TranscriptionJobSummaries[*].TranscriptionJobName]' --output text
while read jobName; do
    aws delete-transcription-job --transcription-job-name "$jobName"
done

However, when this runs, it lists the transcription jobs, but does not delete them. In fact, it does nothing at all after deleting the jobs! Although, it continues to process the job even through it does nothing.

Any ideas on how to fix this or where I am going wrong?

Owen Murray
  • 135
  • 1
  • 8

2 Answers2

2

You need to pipe the standard output of the aws command into the while read loop's standard input. You'll also want to switch to read -r to disable some legacy behavior of the read command, though it probably doesn't matter here.

aws transcribe list-transcription-jobs --max-results 100 --query '[TranscriptionJobSummaries[*].TranscriptionJobName]' --output text |
while read -r jobName; do
    aws delete-transcription-job --transcription-job-name "$jobName"
done

The crucial fix is the | at the end of the first line. Maybe also read up on basic shell pipelines.

Your original command would simply list the resutls to standard output, then sit there and wait for the read command to receive its input from somewhere else (realistically, from your keyboard).

If aws delete-transcription-job can somehow accept a list of jobs, maybe you can do away with the while loop entirely. I'm unfortunately not familiar with this specific command.

tripleee
  • 175,061
  • 34
  • 275
  • 318
  • I have been trying to revert the job name to become a variable in itself for the delete to run off, however it doesn't seem to be working. Even with your new code, the issue still persists, even when I replace the '|' with a ";" – Owen Murray Oct 17 '19 at 14:18
  • Why would you replace the `|` with a `;`? That's back to square one; semicolon is syntactically equivalent to newline. – tripleee Oct 17 '19 at 15:47
  • If I use a ```|``` it doesn't recognise it as code and gives me an error – Owen Murray Oct 17 '19 at 15:50
  • What error? Why do you think it's not recognized as code? – tripleee Oct 17 '19 at 15:51
  • It says ```aws: error: argument command: Invalid choice, valid choices are:``` followed by a large list of possible commands to use – Owen Murray Oct 17 '19 at 15:53
  • Each command would work on their own, but it won't work when combined – Owen Murray Oct 17 '19 at 15:53
  • I have these commands in a text file which I turned into an executable file using ```chmod 700 [file name]``` I am running the file in Terminal on Mac by typing either ```./transcribejobs``` or ```sh transcribejobs``` – Owen Murray Oct 17 '19 at 15:56
  • (If you are using `sh` you are not running Bash, but that's unimportant here. Maybe see [Difference between sh and bash](https://stackoverflow.com/questions/5725296/difference-between-sh-and-bash).) – tripleee Oct 17 '19 at 15:57
  • I can't reproduce your problem. It sounds *vaguely* like you typed the wrong character instead of `|` but this is descending into pure guesswork. If your original commands worked then both the answers here should work fine. – tripleee Oct 17 '19 at 16:00
  • I have tried running it in bash with the ```|``` copied and pasted from your comment and still have the issue, from reading the original code I have added to the question, should this in theory work? – Owen Murray Oct 17 '19 at 16:03
  • I don't have access to AWS from where I am now, but both of these answers hinge on the assumption that your mistake lies in not connecting the first command to the second (and that there are no additional mistakes or misunderstandings). Your comments here reinforce that impression. If you can run `aws foo` then you should be able to run `aws foo | cat` and see the same results. – tripleee Oct 17 '19 at 16:06
  • That's where I think I am having the issue, connecting the commands together. I just mean when I run the commands on their own, they work, however for the ``` aws delete-transcription-job --transcription-job-name "$jobName"``` I need to manually enter the ```jobName```, but I can't do this for all 17000 jobs – Owen Murray Oct 17 '19 at 16:13
0

Simply

IFS="
"
for jobName in `aws transcribe list-transcription-jobs --max-results 100 --query '[TranscriptionJobSummaries[*].TranscriptionJobName]' --output text`; do
    aws delete-transcription-job --transcription-job-name "$jobName"
done
Akhil
  • 912
  • 12
  • 23
  • I tried the following code but was given an error that I had received from a previous version I tried out. The error message is as follows: ```aws: error: argument command: Invalid choice,``` – Owen Murray Oct 17 '19 at 14:17
  • use ```set -x``` at the top and see what's wrong there. – Akhil Oct 18 '19 at 04:59