OS X: equivalent of Linux's wget

Question

How can I do an HTTP GET from a Un*x shell script on a stock OS X system? (installing third-party software is not an option, for this has to run on a lot of different systems which I don't have control on).

For example if I start the Mercurial server locally doing a hg serve:

... $ hg serve

And then, from a Linux that has the wget command I do a wget:

... $  wget http://127.0.0.1:8000
--2010-12-31 22:18:25--  http://127.0.0.1:8000/
Connecting to 127.0.0.1:8000... connected.
HTTP request sent, awaiting response... 200 Script output follows
Length: unspecified [text/html]
Saving to: `index.html

And on the terminal in which I launched the "hg serve" command, I can indeed see that an HTTP GET made its way:

127.0.0.1 - - [30/Dec/2010 22:18:17] "GET / HTTP/1.0" 200 -

So on Linux one way to do an HTTP GET from a shell script is to use wget (if that command is installed of course).

What other ways are there to do the equivalent of a wget? I'm looking, in particular, for something that would work on stock OS X installs.

Also convenient: http://mikebeach.org/2012/09/29/how-to-install-wget-in-mac-os-x/ — emilyk, May 09 '13 at 22:23

SiegeX · Accepted Answer · 2022-10-29T16:41:32.633

889

The following native command will work:

curl http://127.0.0.1:8000 -o outfile

Note that curl does not follow redirects by default. To tell it to do so, add -L to the argument list.

edited Oct 29 '22 at 16:41

answered Dec 31 '10 at 20:21

SiegeX

135,741
24
144
154

27

Also, *wget* is available via both [MacPorts](http://www.macports.org/) and [Fink](http://www.finkproject.org/). – Brian Clapper Dec 31 '10 at 20:26
47

I came here from a Google search on how to get wget on Mac, so despite the OP's requirement to avoid installation of 3rd party software, I'll provide a link to a quick guide I found on [how to install wget without using MacPorts](http://osxdaily.com/2012/05/22/install-wget-mac-os-x/) for future reference. – Eirik H Mar 20 '13 at 08:28
44

It's also available on Homebrew. – Daniel Node.js Oct 10 '14 at 00:42
1

Can I resume downloads using this? – Sajib Acharya Apr 29 '16 at 19:55
4

Doesn't seem to follow forwards. – nroose Jan 02 '17 at 02:12
29

For downloading files you can use `-O` or `--remote-name` flag to auto rename downloaded file. e.g. `curl -O http://somehost.org/file.zip` – nimcap Feb 22 '18 at 10:49
I have thought about aliasing `wget` to `curl -O` on my mac's, but this would be evil because it wouldn't be "real" wget. So I don't do that, and use my brain to remember `curl -O`. – Steven Lu May 06 '18 at 05:16
2

You might want to use `curl -LO`, because `wget` follows redirects. – Imaskar May 07 '20 at 17:19
Remember to use -L to avoid [this problem](https://unix.stackexchange.com/questions/321710/why-wont-curl-download-this-link-when-a-browser-will) – Matias de Andrea Oct 25 '22 at 09:58

Eric Hartford · Answer 2 · 2018-08-17T13:12:55.660

588

brew install wget

Homebrew is a package manager for OSX analogous to yum, apt-get, choco, emerge, etc. Be aware that you will also need to install Xcode and the Command Line Tools. Virtually anyone who uses the command line in OSX will want to install these things anyway.

If you can't or don't want to use homebrew, you could also:

Install wget manually:

curl -# "http://ftp.gnu.org/gnu/wget/wget-1.17.1.tar.xz" -o "wget.tar.xz"
tar xf wget.tar.xz
cd wget-1.17.1
./configure --with-ssl=openssl -with-libssl-prefix=/usr/local/ssl && make -j8 && make install

Or, use a bash alias:

function _wget() { curl "${1}" -o $(basename "${1}") ; };
alias wget='_wget'

edited Aug 17 '18 at 13:12

answered Jun 13 '13 at 18:18

Eric Hartford

16,464
4
33
50

12

You might take a moment to explain homebrew, but it's important that there be a newbie-visible `wget` answer here since the only other one was deleted by it's owner, and `curl` is an alternative rather than a literal equivalent. – Chris Stratton Jun 13 '13 at 18:42
4

Thanks, this was helpful to me as someone with brew already installed. – enderland Jul 13 '13 at 18:48
4

@Michaelangelo And that is not your job to vandalize others' posts. You should not be editing other answers to make your own points - that's inappropriate. In fact, the actions are being discussed on Meta: http://meta.stackoverflow.com/questions/315892/should-i-rollback-an-edit-that-i-reviewed-and-rejected – Zizouz212 Feb 01 '16 at 20:07
2

@EricHartford Down-voting doesn't solve the problem. It provided *one* alternative, without listing the **other** option of how to manually install wget. – Michaelangel007 Feb 01 '16 at 20:38
1

I think you have a good point. And I will edit the answer to incorporate your ideas. – Eric Hartford Feb 01 '16 at 20:46
fixed by updating to version 1.17.1 – Eric Hartford Aug 17 '18 at 13:13
that wget alias is really cool! – Cagri Uysal Apr 07 '22 at 19:42

score 106 · Answer 3 · edited Jan 25 '14 at 11:17

106

Curl has a mode that is almost equivalent to the default wget.

curl -O <url>

This works just like

wget <url>

And, if you like, you can add this to your .bashrc:

alias wget='curl -O'

It's not 100% compatible, but it works for the most common wget usage (IMO)

edited Jan 25 '14 at 11:17

Community

1
1

answered Jan 10 '14 at 08:16

Ed Henderson

1,184
1
7
4

2

how do u do recursive with this? – Jasper Nov 07 '15 at 14:52
You can use bash to add this into a loop like this: `for i in \`seq 1 \` do curl -O ;done;` where is the number of times you want to iterate and is the url to pull. – Blairg23 Dec 15 '15 at 23:32
1

Sometimes you'll need to add the `-L` flag to follow location redirects. You can use `curl -OL ` to do that. – Gustavo Straube Jan 27 '16 at 12:48
1

`-O` also only applies to the next argument, so to download multiple URLs you have to use something like `curl -O "$url1" -O "$url2"` or `printf %s\\n "$url1" "$url2"|xargs -n1 curl -O`. – nisetama Apr 19 '16 at 00:47
It's a great neat tip. Thank you so much! – cmcromance Aug 13 '16 at 16:57

score 44 · Answer 4 · edited Feb 01 '16 at 18:21

44

1) on your mac type

nano /usr/bin/wget

2) paste the following in

#!/bin/bash
curl -L $1 -o $2

3) close then make it executable

chmod 777 /usr/bin/wget

That's it.

edited Feb 01 '16 at 18:21

Zizouz212

4,908
5
42
66

answered Nov 04 '13 at 12:16

Eamon Straughn

467
4
2

1

Better than an alias. – M K Feb 20 '14 at 09:29
4

Almost correct. I believe that step one should be `vim /usr/bin/wget` though ;) haha just kidding. thanks for the answer -- this never really occurred to me and for some reason I don't feel like installing brew/fink/whatever, so kudos for the easy portable answer. – Kasapo Apr 02 '15 at 18:47
4

`-L` is important for handling http `301` responses. wget handles them by default. – Donn Lee May 22 '18 at 17:28
"curl -L https://resource.url/tar.tar.gz -O tar.tar.gz" worked fine, thanks for this sole workable solution among others in this thread for my use case. – Oleksii Kyslytsyn Jul 08 '18 at 12:12
sudo nano /usr/bin/wget – Dec 21 '18 at 21:59
wow. this answer taught me *way* more than I was hoping to get out of this question.. I've spiraled into a deep dive on shell scripting and what not for the past 6 hours... Thanks – Govind Rai Mar 04 '19 at 02:39

score 19 · Answer 5 · answered Dec 31 '10 at 20:22

19

Use curl;

curl http://127.0.0.1:8000 -o index.html

answered Dec 31 '10 at 20:22

ismail

46,010
9
86
95

score 17 · Answer 6 · answered Jun 27 '13 at 07:57

Here's the Mac OS X equivalent of Linux's wget.

For Linux, for instance Ubuntu on an AWS instance, use:

wget http://example.com/textfile.txt

On a Mac, i.e. for local development, use this:

curl http://example.com/textfile.txt -o textfile.txt

The -o parameter is required on a Mac for output into a file instead of on screen. Specify a different target name for renaming the downloaded file.

Use capital -O for renaming with wget. Lowercase -o will specify output file for transfer log.

score 6 · Answer 7 · answered May 17 '18 at 21:06

6

Instead of going with equivalent, you can try "brew install wget" and use wget.

You need to have brew installed in your mac.

answered May 17 '18 at 21:06

Jaya Konduru

195
2
4

4

Not gonna downvote, but this is a dupe of Eric's answered Jun 13 '13. – MarkHu Sep 14 '18 at 02:30

Hammad Haleem · Answer 8 · 2017-01-16T07:34:18.917

5

You can either build wget on the mac machine or use MacPorts to install it directly.

sudo port install wget

This would work like a charm, also you can update to the latest version as soon as it's available. Port is much more stable than brew, although has a lot less number of formula and ports.

You can install MacPorts from https://www.macports.org/install.php you can download the .pkg file and install it.

edited Jan 16 '17 at 07:34

answered Apr 07 '16 at 02:05

Hammad Haleem

1,374
1
16
26

and how do you install port? – Roy Lovejoy Jan 09 '17 at 22:21

score 4 · Answer 9 · answered Dec 31 '10 at 20:22

4

You could use curl instead. It is installed by default into /usr/bin.

answered Dec 31 '10 at 20:22

James Sumners

14,485
10
59
77

score 2 · Answer 10 · answered Jul 10 '18 at 23:58

2

wget Precompiled Mac Binary

For those looking for a quick wget install on Mac, check out Quentin Stafford-Fraser's precompiled binary here, which has been around for over a decade:

https://statusq.org/archives/2008/07/30/1954/

^{_{MD5 for 2008 wget.zip: 24a35d499704eecedd09e0dd52175582}}
^{_{MD5 for 2005 wget.zip: c7b48ec3ff929d9bd28ddb87e1a76ffb}}

No make/install/port/brew/curl junk. Just download, install, and run. Works with Mac OS X 10.3-10.12+.

answered Jul 10 '18 at 23:58

Beejor

8,606
1
41
31

1

Wget has 11 CVEs since 2008, https://www.cvedetails.com/vulnerability-list/vendor_id-72/product_id-332/GNU-Wget.html , so use this advise on your own risk. – mickvav Jan 29 '23 at 22:06

OS X: equivalent of Linux's wget

10 Answers10

wget Precompiled Mac Binary

Linked