0

I'm trying to parse the result of a CURL request. More specific, the "Location" header. Note: I'm trying to execute one HTTP request with CURL and use the following for processing: command exit code, stderr output (HTTP status code & HTTP headers) and stdout output (HTTP body, in this case: HTML).

I'm having some strange issues with string comparisons on the output and therefore I've started echo-ing the output and started to see some inexplicable things.

Bash script (./google_header_location.sh)

#!/bin/bash
HTTP_HEADER_LOCATION=$(curl -I google.com 2>/dev/null | grep Location |  awk '{print $2}')
echo "HTTP header location: START ${HTTP_HEADER_LOCATION} END"

Command to execute the script above

root@1274eaa3c485:/# ./google_header_location.sh

Response

END header location: START http://www.google.com/

I've ran the script above on: https://hub.docker.com/layers/ubuntu/library/ubuntu/20.04/images/sha256-9d42d0e3e57bc067d10a75ee33bdd1a5298e95e5fc3c5d1fce98b455cb879249?context=explore. More generic: ubuntu:20.04 on hub.docker.com

I expected to see: echo "HTTP header location: START http://www.google.com END"

I don't understand why "END" is echo'd first. What am I doing wrong?

slimdusty
  • 1
  • 1
  • 1
    The `curl` output has CR-LF line endings so a CR character before ` END` puts it at the start of the line. See [Are shell scripts sensitive to encoding and line endings?](https://stackoverflow.com/q/39527571/4154375) and [How to convert Windows end of line in Unix end of line (CR/LF to LF)](https://stackoverflow.com/q/3891076/4154375). – pjh Jun 05 '22 at 19:24
  • this may be an issue with an unexpected EOL character. An odd workaround may be to add `| tr '\r' '\n'` in your command pipe. – leu Jun 05 '22 at 19:24
  • 2
    BTW, `grep ... | awk ...` is almost never needed, since `awk` can do everything itself: `awk '/^Location:/ {print $2}'` or `awk '($1=="Location:") {print $2}'`. It can even remove the carriage return character that's causing trouble: `awk '/^Location:/ {sub("\r","",$2);print $2}'` – Gordon Davisson Jun 05 '22 at 19:26

0 Answers0