50

I would like to download a file remotely with this URL using wget:

https://test.mydomain.com/files/myfile.zip

The site test.mydomain.com requires a login. I would like to download that file in my another server using this command but it does not work (does not completely download the file):

wget --user=myusername --password=mypassword https://test.mydomain.com/files/myfile.zip

If my username is myusername and password is mypassword what would be the correct wget syntax?

The following are the return messages after I type the above command:

Resolving test.mydomain.com (test.mydomain.com)... 123.456.789
Connecting to test.mydomain.com (test.mydomain.com)|123.456.789|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://test.mydomain.com/login/unauthorized [following]
--2013-01-30 02:01:32--  https://test.mydomain.com/login/unauthorized
Reusing existing connection to test.mydomain.com:443.
HTTP request sent, awaiting response... 302 Found
Location: https://test.mydomain.com/login [following]
--2013-01-30 02:01:32--  https://test.mydomain.com/login
Reusing existing connection to test.mydomain.com:443.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: `myfile.zip'

Am I missing something? Please help. Thanks.

Emerson Maningo
  • 2,189
  • 9
  • 32
  • 47
  • checkthis out http://stackoverflow.com/questions/4272770/wget-with-authentication – thomasbabuj Jan 30 '13 at 02:16
  • Just did a quick Google search and found [this link.](http://www.gnu.org/software/wget/manual/html_node/HTTPS-_0028SSL_002fTLS_0029-Options.html) It seems you need to add the parameter `--secure-protocol=protocol`, with protocol being ‘auto’, ‘SSLv2’, ‘SSLv3’, or ‘TLSv1’. – Piccolo Jan 30 '13 at 02:16
  • It doesn't completely download the file. So it has downloaded part of the file? – StarPinkER Jan 30 '13 at 02:21
  • I see the file created on my server but it is not the correct file size. Sort of the wget command only created the file but does not download the contents. – Emerson Maningo Jan 30 '13 at 02:24
  • 1
    Take a try on this addon, https://addons.mozilla.org/en-US/firefox/addon/cliget/, it will help you get the correct wget command. – StarPinkER Jan 30 '13 at 02:26
  • Actually cliget is HIGHLY recommended if the answer below does not work. I suggest everyone having an issue with wget should try it. – Emerson Maningo Feb 02 '13 at 12:26
  • @thomasbabuj Page removed. – Rodrigo Sep 14 '20 at 16:31
  • 1
    @Rodrigo It's still on the Wayback Machine - try https://web.archive.org/web/20170818004821/https://stackoverflow.com/questions/4272770/wget-with-authentication – AJM Jul 26 '21 at 15:44

4 Answers4

101

By specifying the option --user and --ask-password wget will ask for the credentials. Below is an example. Change the username and download link to your needs.

wget --user=username --ask-password https://xyz.com/changelog-6.40.txt
thomasbabuj
  • 3,779
  • 3
  • 24
  • 22
  • 25
    +1 for using `--ask-password` instead of providing it in plaintext to be seen in the shell history. – Zoltán Mar 20 '14 at 09:39
  • 14
    if you use wget in a script --ask-password is useless – faster2b Feb 15 '16 at 13:17
  • Thank's And how do I securely store the password on linux, and does this wget authenticate automatically, without having to enter username and password? – Pereira Dec 05 '18 at 21:14
26

I have found that wget does not properly authenticate with some servers, perhaps because it is only HTTP 1.0 compliant. In such cases, curl (which is HTTP 1.1 compliant) usually does the trick:

curl -o <filename-to-save-as> -u <username>:<password> <url>

David Siegal
  • 4,518
  • 2
  • 17
  • 15
  • wget supports HTTP 1.1 [since 2011](http://en.wikipedia.org/wiki/Wget#Notable_releases) – berbt Dec 30 '14 at 22:31
  • 1
    Thanks. No idea what's going on but wget just won't send the headers for me. curl works fine. – musiKk Dec 18 '15 at 09:41
  • @DavidSiegal how to use curl for downloading files from `google drive`. I used the code `curl -u email@gmail.com:pass https://drive.google.com/drive/u/0/my-drive/3521.jpg > 3521.jpg` it download me a file, but it cannot be open? – Kasun Siyambalapitiya Nov 18 '16 at 08:20
  • It be would recommended to specify just the username in command, let it ask the password and then enter it. Just to prevent password being seen in commands history. – basit Sep 14 '17 at 09:35
  • @berbt Your wikipedia link about wget supporting HTTP 1.1 get its source from a terse revision comment ["Mention support for HTTP/1.1"](http://bzr.savannah.gnu.org/lh/wget/trunk/revision/2336). It's not clear whether that means full or only partial support. The official [Wget FAQ](https://www.gnu.org/software/wget/faq.html#Does_Wget_understand_HTTP.2F1.1.3F) says the latter. – David Siegal Oct 24 '17 at 18:46
13

It's not that your file is partially downloaded. It fails authentication and hence downloads e.g "index.html" but it names it myfile.zip (since this is what you want to download).

I followed the link suggested by @thomasbabuj and figured it out eventually.

You should try adding --auth-no-challenge and as @thomasbabuj suggested replace your password entry

I.e

wget --auth-no-challenge --user=myusername --ask-password https://test.mydomain.com/files/myfile.zip
niCk cAMel
  • 869
  • 1
  • 10
  • 26
  • `man wget` on `--auth-no-challenge`: "Use of this option is not recommended, and is intended only to support some few obscure servers, which never send HTTP authentication challenges, but accept unsolicited auth info, say, in addition to form-based authentication." – Rodrigo Sep 14 '20 at 16:34
  • 1
    Yeah, good input. I'm using curl whenever possible, but if wget is a must, the `--auth-no-challenge` is a must in certain automised environments. I don't agree with "obscure server" notation. – niCk cAMel Oct 09 '20 at 07:29
3

You could try the same address with HTTP instead of HTTPS. Be aware that this does use HTTP instead of HTTPS and only some sites might support this method.

Example address: https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-10.3.0-amd64-netinst.iso

wget http://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-10.3.0-amd64-netinst.iso

*notice the http:// instead of https://.

This is probably not recommended though :)

If you can, try use curl.

EDIT:

FYI an example with username (and prompt for password) would be:

curl --user $USERNAME -O http://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-10.3.0-amd64-netinst.iso

Where -O is

 -O, --remote-name
              Write output to a local file named like the remote file we get. (Only the file part of the remote file is used, the path is cut off.)
tinnick
  • 316
  • 1
  • 2
  • 15