0

so basically I want to download all zip files on a given website using wget and I'm having a hard time. I'm new to this so please bear with me. The website DOES NOT have a page that list all the zip files. Is there a way I can have wget go through the entire site like a webcrawler and download all the zip files? I've tried commands like - 

1) wget -r -np -l 1 -A zip http://site/path/

2) wget -A zip -m -p -E -k -K -np http://site/path/

3) wget --no-clobber --convert-links --random-wait -r -p -E -e robots=off -U mozilla http://site/path/

supposedly they search through the entire site, I haven't been getting those results though. Help or pointing me in the right direction would be very much appreciated!

user3296487
  • 326
  • 2
  • 12
  • If these files are not mentioned, `wget` will not see them. From [the manual](http://www.gnu.org/software/wget/manual/wget.html): "Wget can follow links in HTML, XHTML, and CSS pages, to create local versions of remote web sites, fully recreating the directory structure of the original site. This is sometimes referred to as “recursive downloading.”" – Jongware Jun 17 '15 at 22:05
  • What so I need a url to a page with all the zip file links? – user3296487 Jun 17 '15 at 22:08
  • That is basically what that quote seems to say, yes. If the list of files can be accessed through FTP: the same page mentions you can use wildcards for these. – Jongware Jun 17 '15 at 22:10
  • Oh I see thank you very much – user3296487 Jun 17 '15 at 22:44
  • Does this answer your question? [How to download HTTP directory with all files and sub-directories as they appear on the online files/folders list?](https://stackoverflow.com/questions/23446635/how-to-download-http-directory-with-all-files-and-sub-directories-as-they-appear) – Software Engineer Nov 22 '20 at 08:55

0 Answers0