wget: batch download with increment

Question

I have this situation: I want to download a bunch of files named like this:

683482, 684483, 685484, 686485, 687486, 688487, 689488, 690489, 691490, 692491, ...

As you can see, the files are numbered with an increment of 1001. So, what's the easiest way to do a batch download?

score 0 · Answer 1 · answered Apr 08 '17 at 19:41

0

Please try this:

#!/bin/bash  
for i in {683482..1000000..1001}
    do
        wget $i
    done

answered Apr 08 '17 at 19:41

kodmanyagha

932
12
20

or `for ((i=683482; i<=1000000; i=i+1001))` – Cyrus Apr 08 '17 at 19:57
I'm not expert in bash programming. May be this works or not i dont know. – kodmanyagha Apr 08 '17 at 19:59
`wget /$i` Also, wget accepts more than one URL argument. For a faster download, add up the URLs in a variable and then execute wget just once. Just one DNS lookup and likely there will be a persistent connection. – rockdaboot Apr 11 '17 at 13:44

score 0 · Answer 2 · edited May 23 '17 at 10:30

ECHO OFF
CLS
SETLOCAL ENABLEDELAYEDEXPANSION

FOR /L %%A IN (683482,1001, 692491) DO (
    SET stringWget=wget %%A
    !stringWget!
)

I'll go step by step:

ECHO OFF prevents windows command line from displaying every command on the command-prompt (this is optional; But, looks clean).

CLS clear screen; This clears command prompt's console display. It does not clear the temporary environment variables or command history.

SETLOCAL ENABLEDELAYEDEXPANSION is used for non-blocking statements and is Windows specific; When we use %ENVIRONMENTVARIABLE% , we are implying blocking statement and when we use !ENVIRONMENTVARIABLE! we are implying non-blocking statement (meaning, in a loop, we see updated values of %A instead of repeating what %A had when entering loop). We use %% instead of %in batch files.

FOR loop's syntax can be found in reference.

Reference: https://stackoverflow.com/a/3541415

wget: batch download with increment

2 Answers2