Using netcat to issue a http get request in bash

Question

I have written the following script (it does not support index yet). The issue I am having is regarding using netcat to load the page. I can do this manually using the command line but when I try to have my script issue the exact same commands I can it does nothing without error. The only possible thing I can think is that my output is going somewhere else?

#!/bin/bash
PORT=80

while [ true ]
do
    echo "Type the address you wish to visit then, followed by [ENTER]:"
    read address
    path=${address#*/}
    domain=${address%%/*}
    nc $domain $PORT
    printf "GET /$path HTTP/1.1\n" 
    printf "Host: $domain\n" 
    printf "\n" 
    printf "\n" 
done

`while [ true ]` doesn't do what you think it does, though it does actually happen to work. You mean `while true` or `while [ "nonempty-string-is-still-nonempty" ]` — tripleee, Nov 23 '17 at 06:19
Tangentially related: https://stackoverflow.com/questions/36371221/bash-if-statement-too-many-arguments; also https://stackoverflow.com/questions/37586811/pass-commands-as-input-to-another-command-su-ssh-sh-etc isn't really about this, but should hopefully help clarify your mental model of how scripts work. — tripleee, Nov 23 '17 at 06:29

score 0 · Answer 1 · answered Nov 22 '17 at 21:13

You need to pass the request header to the standard input of nc.

One way to do that is input redirection:

while true
do
    echo "Type the address you wish to visit then, followed by [ENTER]:"
    read address
    path=${address#*/}
    domain=${address%%/*}
    nc $domain $PORT <<EOF
GET /$path
Host: $domain

EOF
done

score 0 · Accepted Answer · answered Nov 22 '17 at 21:13

nc works by copying standard input to the given address:port, and copying whatever it reads from there to standard output.

{
  printf "GET /%s HTTP/1.1\r\n" "$path"
  printf "Host: %s\r\n" "$domain"
  printf "\r\n" 
  printf "\r\n"
} | nc "$domain" "$port"

HTTP requires \r\n at the end of each line. Some servers accept just \n but some stick to the letter of the law and accept only \r\n as specified.
You need to send these lines to the standard input of nc.
It is better to use printf "Get %s HTTP/1.1\r\n" "$path" because$pathmay contain%signs and that would confuseprintf`.

Using netcat to issue a http get request in bash

2 Answers2