Multiple brace expansion in wget URL

Question

I am currently trying to scrape images of a website with a little script I've made :

for url in $my_url/{1..100}'.png' 
do
    wget "$url" || break
done

The fast is that sometimes, images are named 1.png or 01.png or 001.png

So I would like to try the download of the images with each name to not miss any pictures.

Something like :

for url in $my_url/{1..100}{01..100}{001..100}'.png'

Thanks for the help !

score 1 · Answer 1 · answered Apr 24 '18 at 19:39

You need to duplicate them:

for url in "$my_url"/{1..100}.png "$my_url"/{01..100}.png "$my_url"/{001..100}.png

or, using an array aids readability

urls=( 
    "$my_url"/{1..100}.png 
    "$my_url"/{01..100}.png 
    "$my_url"/{001..100}.png
)
for url in "${urls[@]}"; do ...

score 1 · Answer 2 · answered Apr 24 '18 at 19:49

1

You can nest brace expansions:

url="http://example.com/"
printf '%s\n' "$url"{{0..9},{00..99},{000..100}}.png

answered Apr 24 '18 at 19:49

that other guy

score 0 · Accepted Answer · answered Apr 24 '18 at 18:48

Try wrapping everything in a for loop that creates multiple zero-padding formats:

for format in %01g %02g %03g
  do for num in $(seq -f $format 1 100)
    do wget ${my_url}/${num}.png || break  
  done  
done

You could also use printf, but would need to use different syntax, as shown here.

3 Answers3