4

I am using https://www.npmjs.com/package/s3 package to download folder from s3 but I am not able to find any method to cancel the download or interrupt the download process.

So how can I interrupt the ongoing download or stop the download process? Any suggestions for packages offering this feature?

Folder size is upto 10GB and folder contains 1000s of file so it is creating multiple requests for same folder.

Varis Bhalala
  • 123
  • 11
  • 2
    With that module, it seems impossible (based on current source). Have use tried the official [`aws-sdk`](https://aws.amazon.com/sdk-for-node-js/) ? `AWS.S3` class has [`getObject`](https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html#getObject-property) function, which returns an [`AWS.Request`](https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/Request.html), which can be aborted – pergy Mar 05 '19 at 16:36
  • Yea but folder size is of upto 10 GB, and I need progress amount and interrupt feature as well. After cancelling download, if I start download then it should calculate file hash to download the left files. @pergy – Varis Bhalala Mar 06 '19 at 05:18
  • I assume you have a connection that usually have .close() function. Or a request object usually have a .abort() function. Have you tried that? – Gillsoft AB Mar 06 '19 at 21:59
  • @GillsoftAB, Yea we have that methods but at a time we have multiple downloading processes, so we can not abort request. directly – Varis Bhalala Mar 07 '19 at 05:35

1 Answers1

8

I modified the code of the s3 module to allow cancellation.

Here's the PR, but in the meantime you can use my fork for testing:

const downloader = client.downloadFile(params);

downloader.on('error', function(err) {
  console.error('unable to download:', err.stack);
});

downloader.on('cancelled', function() {
  console.log('Download was cancelled:');
});

downloader.on('progress', function() {
  console.log('progress', downloader.progressAmount, downloader.progressTotal);
});

downloader.on('end', function() {
  console.log('done downloading');
});

setTimeout(() => {
  downloader.emit('cancel');
}, 2000);

What I added is a way to call request.abort() on the s3.getObject request.

Marcos Casagrande
  • 37,983
  • 8
  • 84
  • 98
  • this is fine but there is one problem, if I am cancelling one downloading process and just after that I am starting that same downloading process again then it is not working properly. – Varis Bhalala Mar 11 '19 at 03:53
  • How is it not working properly? I tested exactly that, can you show what are you doing, or give a snippet I can try? This is a cancel, not a pause functionality. You have to create a new download. – Marcos Casagrande Mar 11 '19 at 03:56
  • It works great when I call the `downloadFile` method, but in my use-case we need to download an entire directory, so I call the method `downloadDir` but the abort process doesn't work for directories – Varis Bhalala Mar 15 '19 at 07:12
  • Well, you should've said that you're using downloadDir method. That's why questions should have code. I'll try to check that method and implement a cancel too. – Marcos Casagrande Mar 15 '19 at 10:59