Show the final redirect headers using curl

Question

The curl command:

curl -IL "http://www.bbc.co.uk"

returns:

HTTP/1.1 301 Moved Permanently
Server: nginx
X-BBC-No-Scheme-Rewrite: 1
X-Cache-Action: HIT
X-Cache-Hits: 18686
Vary: X-BBC-Edge-Scheme
Cache-Control: public, max-age=3600
X-Cache-Age: 2624
Content-Type: text/html
Date: Thu, 02 Nov 2017 16:55:53 GMT
Location: https://www.bbc.co.uk/
Content-Length: 178
Connection: Keep-Alive

HTTP/1.1 200 OK
Content-Type: text/html; charset=utf-8
ETag: W/"3fdf4-XNfGeb/Aqcmv8OYZKVQdoUFYC34"
X-Frame-Options: SAMEORIGIN
Content-Length: 261620
Date: Thu, 02 Nov 2017 16:55:53 GMT
Connection: keep-alive
Set-Cookie: BBC-UID=65699f7bb41e91e9d7e103d8618f7780367bf9e637843476ca70678244f407000curl/7.19.7%20(x86_64-redhat-linux-gnu)%20libcurl/7.19.7%20NSS/3.19.1%20Basic%20ECC%20zlib/1.2.3%20libidn/1.18%20libssh2/1.4.2; expires=Mon, 01-Nov-21 16:55:53 GMT; path=/; domain=.bbc.co.uk
X-Cache-Action: HIT
X-Cache-Hits: 5
X-Cache-Age: 0
Cache-Control: private, max-age=0, must-revalidate
Vary: Accept-Encoding, X-CDN, X-BBC-Edge-Scheme

(In production, I use -i and not -I to get the whole payload)

How do I get curl to hide the preceding header(s) that issue the redirect, and just return the header of the final target resource?

score 3 · Accepted Answer · answered Nov 02 '17 at 23:06

There's no such an option in curl, but this simple sed program is all you need:

sed '/^HTTP\/1.1 3[0-9][0-9]/,/^\r$/d' file

If you pipe your curl output through it, it will delete (notice the d action at the end) all header blocks that start with any 3xx redirection code (HTTP/1.1 3[0-9][0-9]) and end with an empty line (an empty line in Unix is actually \n only, recognized in sed with ^$ pattern, but HTTP headers use DOS newline sequence \r\n, which after stripping by sed becomes \r; so we have to match ^\r$). To select a range of lines, sed range addressing is used with regexp addresses.

For your example:

$ curl -sIL "http://www.bbc.co.uk" | sed '/^HTTP\/1.1 3[0-9][0-9]/,/^\r$/d'
HTTP/1.1 200 OK
Content-Type: text/html; charset=utf-8
ETag: W/"3fbed-SZN5fj1ugmv3uhAJWfQBrAvzK1M"
X-Frame-Options: SAMEORIGIN
Content-Length: 261101
Date: Thu, 02 Nov 2017 22:53:19 GMT
Connection: keep-alive
Set-Cookie: BBC-UID=95994f2b2a4bf...57f26d5442240curl/7.50.1; expires=Mon, 01-Nov-21 22:53:19 GMT; path=/; domain=.bbc.co.uk
X-Cache-Action: HIT
X-Cache-Hits: 649
X-Cache-Age: 119
Cache-Control: private, max-age=0, must-revalidate
Vary: Accept-Encoding, X-CDN, X-BBC-Edge-Scheme

Similarly for the complete response (with body):

$ curl -siL "http://www.bbc.co.uk" | sed '/^HTTP\/1.1 3[0-9][0-9]/,/^\r$/d'

Notice the -s flag we used above to suppress curl's progress output.

Word of warning to anyone who reads this (spent last 2 hours on this!) if using POST, curl will only send the data payload to the first URL. The redirect URLs will only be GETs (https://stackoverflow.com/questions/33802582/php-redirection-with-code-307-changes-method-from-post-to-get) — KolonUK, Nov 03 '17 at 14:32
Yes, it says exactly so in the [`-L` docs](https://curl.haxx.se/docs/manpage.html#-L). — randomir, Nov 03 '17 at 14:37

Show the final redirect headers using curl

1 Answers1