Download multiple files simultaneously (parallel) with custom filenames

Question

In a bash script, I am trying to download multiple files in parallel, with custom filenames using a single command (no loops).

I tried using aria2c:

aria2c -j2 URL1 URL2                # BAD: outputs to a single file

aria2c -j2 -Z URL1 -o 1 URL2 -o 2   # BAD: filenames taken from link (-o is ignored)

The second one ignores the output filename because, quoting the aria2c manpage:

In Metalink or BitTorrent download you cannot specify file name. The file name specified here is only used when the URIs fed to aria2 are done by command line without --input-file, --force-sequential option. For example:

$ aria2c -o myfile.zip "http://example1.com/file.zip" "http://example2.com/file.zip"

This is what I want to avoid:

aria2c URL1 -o 1 &
aria2c URL2 -o 2 &
aria2c URL3 -o 3                     # BAD: slow and ugly, because aria2c is called thrice

Any suggestions?

Alternatively, you can download the files and then rename them. However, input file allows you to specify the names of output files. — Alexey Ivanov, Nov 25 '16 at 17:36

score 24 · Answer 1 · answered Jun 11 '19 at 11:48

24

with -Z option:

-Z, --force-sequential[=true|false] Fetch URIs in the command-line sequentially and download each URI in a separate session, like the usual command-line download utilities.

so in your case:

aria2c -Z URL1 URL2 URL3 URL4

answered Jun 11 '19 at 11:48

gulp79

241
2
2

1

This answers the question better IMO. – rkok Dec 18 '19 at 08:06
But why is this needed? – Minh Nghĩa Jan 25 '22 at 01:06
1

@MinhNghĩa I think because it by default considers the URLs to be multiple sources for the same file – FLeX Jan 06 '23 at 17:03
Because `--force-sequential=false` is the default. – Christian Herenz Mar 23 '23 at 15:23

score 21 · Accepted Answer · edited Apr 24 '17 at 00:13

21

Aria2c supports getting URIs from a file.

Try writing your file names into the file and then running "aria2c -i uri-list.txt" or write them to stdout and pipe them to "aria2c -i -"

edited Apr 24 '17 at 00:13

Atemu

295
3
12

answered Nov 23 '13 at 00:56

ArtemB

3,496
17
18

Problem is that my URL's contain variables. – octosquidopus Nov 23 '13 at 01:01
3

Do you mean shell variables? You can still use -i. Something like this: # (echo "http://foo/bar/file-$VAR1.txt"; echo "http://bar/baz/file-$VAR2.txt") | aria2c -i - – ArtemB Nov 23 '13 at 01:04
While aria2c -i- is good, how do you end the input. I tried Ctrl+D but it doesn't end it as far as I see. – Plato Aug 31 '19 at 15:04

Download multiple files simultaneously (parallel) with custom filenames

2 Answers2