What is the proper way to listen to HTTP requests?

Question

I've written a very simple, rather low level HTTP (well, part of HTTP) server as an exercise to get more familiar with this whole web thing I've been avoiding all the time. It works reasonably well for a first attempt (not so much that I would recommend anyone to actually use it, but it does what I tell it to). Now the problem is that the GET operation fails a lot (refreshing helps, but it's not very nice - details below), I assume this is because of the way I read requests (I am fairly certain my routes work):

void start()
{
    //...
    try(ServerSocket webSock = new ServerSocket(47000) {
    //...
        while(running) {
           try {
               Socket sock = webSock.accept();
               //read/write from/to sock 
           }
           //...
           Thread.sleep(10);
        }
    } 
}

(full code here: http://pastebin.com/5B1ZuusH )

I am not sure exactly what I'm doing wrong though.

I do get the error:

This webpage is not available
The webpage at http://localhost:47000/ might be temporarily down or it may have moved permanently to a new web address.
Error 15 (net::ERR_SOCKET_NOT_CONNECTED): Unknown error.

quite a bit (entire page doesn't load), sometimes scripts or images don't load either. If required, I could post the entire code, but the rest is mostly boilerplate.

`in.ready()` tells you if there is data already in the buffer right now. If there isn't because it hasn't reached your tcp stack yet (but is in-flight), it will return false and your whole logic breaks. — Mat, Nov 25 '12 at 14:18
@Mat I see. I figured it would be something like that. Unfortunately I'm not very experienced with this - could you please tell me what common ways to avoid this are? — Cubic, Nov 25 '12 at 14:21
Usually, you use a proper http server. It's actually not trivial to implement, you need to parse the request as it is coming in to know when it is complete. Plus handling persistent connections correctly isn't entirely trivial either. — Mat, Nov 25 '12 at 14:26
@Mat I think writing your own web server much more useful for exploring how web than using some existing server. — Mikita Belahlazau, Nov 25 '12 at 14:31
@Cubic you can check this question to read all data request from stream: http://stackoverflow.com/questions/309424/read-convert-an-inputstream-to-a-string — Mikita Belahlazau, Nov 25 '12 at 14:37

Guido Simone · Answer 1 · 2016-08-11T23:15:42.067

[another update]

OK to clarify my response, here is a trivial web server that shows how to read GET requests. Note that it handles multiple requests in the same connection. If the connection closes, the program exits. Typically though I can send a number of request from the same web browser before the connection closes and the program exits. This means you cannot use end-of-stream as a signal the the message is over.

Please note that I never use a hand-written web-server for anything real. My favorite is Tomcat, but the other frameworks are fine too.

public class MyWebServer
{
   public static void main(String[] args) throws Exception
   {
      ServerSocket server = new ServerSocket(47000);
      Socket conn = server.accept();
      BufferedReader reader = new BufferedReader(new InputStreamReader(conn.getInputStream()));

      // don't use buffered writer because we need to write both "text" and "binary"
      OutputStream out = conn.getOutputStream();
      int count = 0;
      while (true)
      {
         count++;
         String line = reader.readLine();
         if (line == null)
         {
            System.out.println("Connection closed");
            break;
         }
         System.out.println("" + count + ": " + line);
         if (line.equals(""))
         {
            System.out.println("Writing response...");

            // need to construct response bytes first
            byte [] response = "<html><body>Hello World</body></html>".getBytes("ASCII");

            String statusLine = "HTTP/1.1 200 OK\r\n";
            out.write(statusLine.getBytes("ASCII"));

            String contentLength = "Content-Length: " + response.length + "\r\n";
            out.write(contentLength.getBytes("ASCII"));

            // signal end of headers
            out.write( "\r\n".getBytes("ASCII"));

            // write actual response and flush
            out.write(response);
            out.flush();
         }
      }
   }
}

[origin response]

What is the proper way to listen to HTTP requests?

For most of us, the proper way is to use a well designed web server framework such as Tomcat, Jetty, or Netty

as an exercise to get more familiar with this whole web thing

However if this is an academic exercise to learn about HTTP, then the first thing to do is study the HTTP protocol (see http://www.w3.org/Protocols/rfc2616/rfc2616.html). I'm pretty sure you have not done this because you code is making no attempt to identify the start line, headers etc to figure out when the GET request is complete and it make sense to send a response.

[update]

Cool. You've learned about how TCP is stream oriented and does not preserve message boundaries. Yes the application has to handle that. Here is a final thought - you could probably get your experiment to work fairly reliably - only for GET requests mind you- if you used readLine to read the start line and headers. When you get a blank line, the request is done. This will cause the buffered reader to block at the right times so you get all your content.

This will not work for POST etc because you would then need to parse the Content-Length header and read some number of bytes.

Hopefully this experiment will make you appreciate Jetty more when you realize how much is involved in doing this correctly and reliably - so I think it's a worthwhile effort.

I kind of assumed I'd have the entire request available on the sockets `InputStream`, or nothing at all. The parsing is done by the request class. Apparently socket io doesn't work that way. I've skimmed over RFC2616, but this was - as you put it - an academic exercise from the start, so I never intended to implement the whole thing. I've pretty much settled on using Jetty for my project, I just thought I'd try to implement a kind-of working solution to get a feel for what's going on "under the hood" if you so will. — Cubic, Nov 25 '12 at 14:45
@Cubic request available from `InputStream`. You shouldn't use `ready` method to check if it has more data. You need to read data from stream/reader using `read` method until you get -1. From doc: `read returns -1 if the end of the stream has been reached ` — Mikita Belahlazau, Nov 25 '12 at 14:53
@NikitaBeloglazov I don't think it works that way. I'm not getting a -1. — Cubic, Nov 25 '12 at 15:36
@NikitaBeloglazov Nothing. The stream just ends and I block for all eternity. Apparently this is because of the way TCP does (not) handle message boundaries. — Cubic, Nov 25 '12 at 15:46
@Cubic hm, sorry, seems I don't understand how it fully works :( — Mikita Belahlazau, Nov 25 '12 at 16:11
@Cubic see my update. For GET requests call readLine until you get a blank line. You don't want an end-of-stream indicator because that would mean the connection is closed and you will probably fail when you try to send your response. The socket should not close until after you send your response - even then the browser may choose to keep connection open and use it to send the next GET — Guido Simone, Nov 25 '12 at 16:20
@Cubic Yes. See http://docs.oracle.com/javase/6/docs/api/java/io/BufferedReader.html#readLine() — Guido Simone, Nov 25 '12 at 18:18
@GuidoSimone I know this was created in 2012, but I had a question. Is there any particular reason why you end it with `out.flush();;` with two semicolons, instead of just one? — Schwaitz, Aug 08 '16 at 18:54
@GuidoSimone Just know that your answer here has been helpful even 4 years later! — Schwaitz, Aug 12 '16 at 00:27

What is the proper way to listen to HTTP requests?

1 Answers1

Linked