Don't download an existing file with axel

Question

I have an script that reads some urls and get them to axel to download them. I want stop script on sometimes and resume it further. Axel can resumes file downloading. So if I press Ctrl+C when downloading, the next time, it starts from the resume of file.

but axel doesn't check that the file existed. So it appends ".0" to the end of file name and start downloding it twice. How I can tell to axel if the file with same name existed, skip that and don't downloading it??

axel uses an [FILE_NAME].st besides the main downloading file which has the state of downloaded file, if you resume it finds the state file and fetches the next bytes — Hojat Taheri, Mar 20 '14 at 20:28

hkoosha · Answer 1 · 2022-10-11T10:59:36.843

Update, some years later:

If you're using a newer axel, then please simply use the no-clobber option -c, instead of implementing it manually as described here!

If you want to make sure axel will resume, as it does per file name and not per url, you should use a deterministic name for the file:

axel -o NAME_OF_EXISTING_FILE

If you want to check if file exists

if [ -f $FILE ]; then
   echo "File $FILE exists."
    # operation related to when file exists, aka skip download
else
   echo "File $FILE does not exist." 
   # operation related to when file does not exists
fi

In case of axel, you want to start download if...

You do not have that file localy, or

You have a partial download, so

 function custom_axel() {

 local file_thingy="$1"
 local url="$2"
 if [ ! -e "$file_thingy" ]; then
     echo "file not found, downloading: $file_thingy"
     axel -avn8 "$url" -o "$file_thingy"
 elif [ -e "${file_thingy}.st" ]; then
     echo "found partial download, resuming: $file_thingy"
     axel -avn8  "$url" -o "$file_thingy"
 else
     echo "already have the file, skipped: $file_thingy"
 fi

 }

This could go into ~/.bashrc or into /usr/bin/custom_axel.sh and later:

while read URL; do
    name=$(basename "$URL") # but make sure it's valid.
    custom_axel "$name" "$URL"
done < /my/list/of/files.txt

This is the correct answer now, with the caveat of that the `-c` option will ONLY resume download correctly when a `.st` file is present in the same folder. This file is created by axel during the download and is removed once it is completed. If the `.st` file is not there, axel will do nothing to verify the file before concluding `already there; not retrieving.`. — Unverified Contact, Aug 18 '23 at 12:05

score 5 · Answer 2 · edited Mar 27 '16 at 09:14

5

Axel use a state file, named with a .st extension.
It is regularly updated during download.
When axel is started, it first checks for <file> and <file.st>. If found, download is resumed where it stopped.
Which version do you have ? I got Axel version 2.4 (Linux) and it resume correctly after a CTRL+C.

edited Mar 27 '16 at 09:14

Mr Lister

45,515
15
108
150

answered Mar 18 '16 at 16:23

Setop

2,262
13
28

score 1 · Answer 3 · answered Oct 10 '22 at 06:31

1

You can use axel with -c flag. From axel --help:

--no-clobber        -c  Skip download if file already exists

(Version 2.17.11)

answered Oct 10 '22 at 06:31

SaTa

2,422
2
14
26

Don't download an existing file with axel

3 Answers3