1

I am trying to download files using an input file (a.txt) which has URLs using the following commands

wget -i a.txt

URLs are like

https://domian.com/abc?api=123&xyz=323&title=newFile12
https://domian.com/abc?api=1243&xyz=3223&title=newFile13

I want to set the name of the file from the URL by using the title tag (for example in the above URL name of the file download need to be newFile12) but can't find any way around it.

In order to get it done, I have to write a python script (similar to this answer https://stackoverflow.com/a/28313383/10549469) and run one by one is there any other way around.

Seaver Olson
  • 450
  • 3
  • 16
ooo
  • 512
  • 1
  • 7
  • 27
  • Please show what you’ve tried so far and where you’ve gotten stuck. – S3DEV Jul 09 '20 at 15:31
  • I am not stuck but looking if someone knows a better way to do it without python script i.e., directly from `wget` command only. (I used the script from this answer https://stackoverflow.com/a/28313383/10549469) – ooo Jul 09 '20 at 15:34
  • there is no way to do this with only wget but I included a script that should work – Seaver Olson Jul 09 '20 at 15:41

2 Answers2

1

You can create a script on the fly and and pipe it to bash. A bit slower than wget -i but would preserve file names:

 sed "s/\(.*title=\(.*\)\)/wget -O '\2' '\1'/" a.txt

When you are satisfied with the results, you can pipe it to bash:

sed "s/\(.*title=\(.*\)\)/wget -O '\2' '\1'/" a.txt | bash
Sorin
  • 5,201
  • 2
  • 18
  • 45
-1

Have a look at wget --content-disposition or for loop with wget -O <outputfile name> <url>

Following command downloads the file with filename as provided by server (vim-readonly-1.1.tar.gz) instead of download_script.php?src_id=27233.

wget --content-disposition https://www.vim.org/scripts/download_script.php?src_id=27233
Vijay
  • 778
  • 4
  • 9
  • It doesn't work with the long URLs, I have the URLs from youtube video it still gets a bad name. – ooo Jul 09 '20 at 16:12
  • --content-disposition works if server supports it. Other option is you provide name with -O flag in wget command – Vijay Jul 09 '20 at 16:16