NodeJS - TCP - Sending an HTTP request

Question

I'm trying to send an HTTP request via TCP sockets.

But I'm not getting any response from www.google.com at all. No idea what I'm doing wrong.

Here is the code:

var client, net, raw_request;

net = require('net');

raw_request = "GET http://www.google.com/ HTTP/1.1\nUser-Agent: Mozilla 5.0\nhost: www.google.com\nCookie: \ncontent-length: 0\nConnection: keep-alive";

client = new net.Socket();

client.connect(80, "www.google.com", function() {
  console.log("Sending request");
  return client.write(raw_request);
});

client.on("data", function(data) {
  console.log("Data");
  return console.log(data);
});

Hope someone can help me.

Just to clarify... the requst was missing two ending newlines and all newlines had to be in the format of /r/n.

Thanks everyone! :)

Why do you send `\n` and not `\r\n` as the specs require? You also need to finish the request by sending two `\r\n`'s. Do you want to manually build a request or can you use [`http.request()`](http://nodejs.org/api/http.html#http_http_request_options_callback) instead? — CodeCaster, Jan 19 '13 at 18:54
http://stackoverflow.com/questions/9577611/http-get-request-in-node-js-express/9577651#9577651 — bryanmac, Jan 19 '13 at 18:55
Just wanted to write some HTTP requests using TCP sockets to learn something new. — RadiantHex, Jan 19 '13 at 18:56
@bryanmac thanks for the link. But really want to learn how to do this at a lower level. — RadiantHex, Jan 19 '13 at 18:57
@RadiantHex then you should start looking at RFC 2616, especially [section 5, Request](http://tools.ietf.org/html/rfc2616#section-5), how to construct valid HTTP messages. — CodeCaster, Jan 19 '13 at 18:57
@CodeCaster: thanks for the reply. I've changed the newlines in the message and I've remove all headers except for host. Still no reply. Not even an error message — RadiantHex, Jan 19 '13 at 19:03

Florin Petriuc · Accepted Answer · 2013-01-19T19:15:49.133

If you have google chrome installed you can see the exact get request that is sent to google. This is how mine looks like:

GET https://www.google.com/ HTTP/1.1
:host: www.google.com
accept-charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
accept-encoding: gzip,deflate,sdch
accept-language: en-US,en;q=0.8
user-agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.52 Safari/537.17
:path: /
accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
:version: HTTP/1.1
cache-control: max-age=0
cookie: <lots of chars here>
:scheme: https
x-chrome-variations: CMq1yQEIjLbJAQiYtskBCKW2yQEIp7bJAQiptskBCLa2yQEI14PKAQ==
:method: GET

At a first view I can see that chrome is sending the request to https://www.google.com and you are sending to http://www.google.com

Another thing is that you are using "\n" and you need to use "\r\n", and the request has to end with "\r\n\r\n".

If you still can't get any response try using http://77.214.52.152/ instead of http://google.com.

score 1 · Answer 2 · answered Jan 19 '13 at 19:04

 GET /

The way you wrote it, you are trying get something like http://www.google.com/http://www.google.com/

And you need two new-lines at the end, so the web-server knows you're done. That's why you're getting nothing back -- it's still waiting for you.

Always try these things with telnet first:

$ telnet www.google.com 80
GET http://www.google.com/ HTTP

HTTP/1.0 400 Bad Request
Content-Type: text/html; charset=UTF-8
Content-Length: 925
Date: Sat, 19 Jan 2013 19:02:06 GMT
Server: GFE/2.0

<!DOCTYPE html>
<html lang=en>
  <meta charset=utf-8>
  <meta name=viewport content="initial-scale=1, minimum-scale=1, width=device-width">
  <title>Error 400 (Bad Request)!!1</title>
  <style>
    *{margin:0;padding:0}html,code{font:15px/22px arial,sans-serif}html{background:#fff;color:#222;padding:15px}body{margin:7% auto 0;max-width:390px;min-height:180px;padding:30px 0 15px}* > body{background:url(//www.google.com/images/errors/robot.png) 100% 5px no-repeat;padding-right:205px}p{margin:11px 0 22px;overflow:hidden}ins{color:#777;text-decoration:none}a img{border:0}@media screen and (max-width:772px){body{background:none;margin-top:0;max-width:none;padding-right:0}}
  </style>
  <a href=//www.google.com/><img src=//www.google.com/images/errors/logo_sm.gif alt=Google></a>
  <p><b>400.</b> <ins>That’s an error.</ins>
  <p>Your client has issued a malformed or illegal request.  <ins>That’s all we know.</ins>
Connection closed by foreign host.

Thanks a lot for the information. But I'm not getting any reply at all from Google. At least via `client.on('data', ...)` I'm not. — RadiantHex, Jan 19 '13 at 19:08
"The way you wrote it, you are trying get something like http://www.google.com/http://www.google.com/" - that is wrong. The http get request supports absolute URI. Moreover, using absolute URI is the only way to get a http get request to work with proxy. — Florin Petriuc, Jan 19 '13 at 19:11
@FlorinPetriuc -- I don't know what to tell you. I used the absolute URI with Google and, as you see, I got a 400 error, Malformed Request. Call Sergei Brin and take it up with him. In the meanwhile, I would use the HOST: header to get proxies to work. — Michael Lorton, Jan 19 '13 at 19:15

NodeJS - TCP - Sending an HTTP request

2 Answers2