2

I would like to write simple script which will check URLs declared in array and display the result in the format below:

http://google.de : 301 Moved Permanently (http://www.google.de/)
http://google.fr : 301 Moved Permanently (http://www.google.fr/)
https://stackoverflow.com : 200 OK

Here is my script:

#!/usr/bin/env bash

sites=(
    http://google.de
    http://google.fr
    https://stackoverflow.com
)

for site in ${sites[*]}; do
    headers=$(curl -Is ${site})

    status=$(echo "$headers" | head -n 1 | cut -c10-)
    location=$(echo "$headers" | grep 'Location' | cut -c10-)

    echo -n "$site : "
    echo -n "$status"

    if [[ ! -z "$location" ]]; then
        echo -n " ($location)"
    fi

    echo -e "\n"
done;

Anyway it returns the output:

)( http://www.google.de/Moved Permanently

)( http://www.google.fr/oved Permanently

https://stackoverflow.com : 200 OK

I do not understand why those strings are overwritten in such weird way.

EDIT:

I have uploaded the result of bash -x myfile – https://pastebin.com/guMJC24h

Michał Szczech
  • 466
  • 4
  • 17
  • 1
    This means there's a carriage return character in your string. Maybe you created it with a DOS/Windows text editor? – Charles Duffy Feb 23 '18 at 22:02
  • 1
    See the section *Before asking about problematic code* in https://stackoverflow.com/tags/bash/info; this is literally the **very first thing** it tells you to check for. – Charles Duffy Feb 23 '18 at 22:02
  • 1
    BTW, once you've fixed the CRLFs, your loop should be `for site in "${sites[@]}"` (with the quotes), `headers=$(curl -Is "$site")` (the curly braces are optional for correctness, the quotes aren't). Also, `echo -n` and `echo -e` are both best avoided -- see the [POSIX standard for `echo`](http://pubs.opengroup.org/onlinepubs/9699919799/utilities/echo.html), particularly the APPLICATION USAGE section, telling you to use `printf` instead. – Charles Duffy Feb 23 '18 at 22:04
  • My file has LF line break. It was created using Vim on Debian. – Michał Szczech Feb 23 '18 at 22:06
  • Something clearly doesn't. Execute your code with `bash -x yourscript` and it'll log what it's doing in such a manner as to make that "something" clear. – Charles Duffy Feb 23 '18 at 22:07
  • BTW, I'd tend to suggest doing the printing all at once; ie. `printf '%s : %s%s\n' "$site" "$status" "${location+ ($location)}"`; if nothing else, it'll make the `set -x` logs easier to read. – Charles Duffy Feb 23 '18 at 22:09
  • 3
    ...if it turns out there are literal CRs being returned from `curl`, by the way, those'll be easy to remove with `status=${status%$'\r'}` or similar; let me know if that's where the problem is and I'll adjust the duplicate link to a more appropriate question. – Charles Duffy Feb 23 '18 at 22:10
  • Thank you @CharlesDuffy, adding `status=${status%$'\r'}` fixed my problem! – Michał Szczech Feb 23 '18 at 22:14
  • 1
    Glad to hear! I added [printing output from grep in bash yields broken string](https://stackoverflow.com/questions/34137780/printing-output-from-grep-in-bash-script-yields-broken-string), which is also asking about this issue when using `echo` to print content retrieved from headers returned by `curl`. – Charles Duffy Feb 23 '18 at 22:19
  • @MichałSzczech You may use an `awk` *one-liner* too like `curl -Is "$site" | awk -v RS="\r\n" -v site="$site" 'NR==1{$1="";var=site " :" $0}/^Location:/{var=var "(" $2 ")"}END{print var}'` – sjsam Feb 23 '18 at 22:40

0 Answers0