0

Suppose an operation is executed from the cli, will the process continue to run or cease when the user cancels it and closes their CLI/terminal?

Example

E.g. Apparently counting ~50 million files in a bucket takes around an hour. If the user ran the operation (i.e. aws s3 ls s3://mybucket/ --recursive | wc -l), then canceled the operation and closed their terminal, would the operation continue to run (and therefore consume resources), or would it cease as soon as the command is cancelled on the user's CLI?

What I know so far

I do not know a lot, but understand that some commands, when executed, cannot be stopped (even if you close the Terminal) - that is, they're running 'detached'

stevec
  • 41,291
  • 27
  • 223
  • 311

1 Answers1

1

Indeed, commands line 

aws s3 ls s3://mybucket/ --recursive

will start running, but the other one,

wc -l

will not execute if the command is terminated.

You may have a bash script or multiple aws commands running in pipe and they run 1 by 1. If you interrupt, if will finish only current one and not start any new ones.

Think of them as commands/buttons, you pressed the ls button and told AWS S3 to list, that is done, S3 will continue to list and it finishes.

EDIT

Missed the large amount of files, my answer is available for 1000 (default page size) or less items. You can configure the page size, but maximum is 1000, more info on this page.

For every page, local CLI will send a new request to S3. When you interrupt, the page will still be completely executed by AWS, but further pages will not be triggered.

Horatiu Jeflea
  • 7,256
  • 6
  • 38
  • 67
  • Sorry, but that is not at all how this works. The aws CLI runs very much in the foreground. Nothing either locally or at AWS is doing any background work in the sense that you have described. `aws s3 ls` for a large bucket is continually sending API requests to the bucket for the next 1000 objects, and the next 1000 and the next 1000, and when you stop the local aws-cli processs, all of that activity stops. – Michael - sqlbot Oct 11 '19 at 23:57