0

I wanted to download all audio content from a website with articles so I could listen to it at higher speed on my phone. I created a script that I can paste in the browser inspector tools to trigger that download. It gets all downloadable content, adds a download attribute with the name of the file and triggers a click to save. I ran into a problem, Chrome has a maximum simultaneous download limit and does not queue things on its own. So after around 19 files the downloads will silently fail. I found a way around this, every X downloads that I trigger I wait (blocking) for Y seconds to give time for the other downloads to complete.

Here is the code for that:

// wait function taken from https://stackoverflow.com/a/33414145
function wait(ms) {
    var start = new Date().getTime();
    var end = start;
    while (end < start + ms) {
        end = new Date().getTime();
    }
}
let count = 0;

function downloadAudio(downloadClass) {
    jQuery(downloadClass).each(function (index) {
        count++;
        if (count > 10) { // wait every few downloads because we have a limit of simultaneous downloads
            console.log('paused before ' + index);
            wait(30000);
            console.log('paused after' + index);
            count = 0;
        }

        let $downloadElement = jQuery(this);


        // Add attribute download to make the file auto downloadable, and put the name of the file as the URL
        $downloadElement.attr("download", $downloadElement.attr("href"));
        // the reason why get(0) is needed and you can't simply trigger click on the jQuery element is explained here https://stackoverflow.com/a/25507433
        $downloadElement.get(0).click(); // the reason why this is needed https://stackoverflow.com/questions/25507314/click-on-download-link-using-jquery-doesnt-download-file
    });
}

downloadAudio('.audio-member__download-article');

This solution works fine, although it could have easily failed if the downloads don't finish fast enough. It is however slow (not a big issue) and inelegant.

Is there a more elegant way to handle this throttling? My interest is purely academic as the script I created does the job. Would be great if that better way was not only nicer looking but also was faster (mine has unnecessary waiting to play it safe) and safer (mine could still miss downloads).

pcatre
  • 1,304
  • 2
  • 16
  • 24
  • 1
    Download the file as a file blob. Then save that. That way you can do it one at a time. You can also use promises to sleep and await. – John Dec 24 '20 at 11:22
  • that saving part where I can check the progression of the download without a library and know when it was finished is something I'm missing. Do you have any link to an example? – pcatre Dec 24 '20 at 11:29
  • 1
    https://javascript.info/fetch-progress – John Dec 24 '20 at 11:44
  • great :) thank you! – pcatre Dec 24 '20 at 12:26

0 Answers0