Java HTTP GET response waits until timeout

Question

I am supposed to develop a program that reads a web page with the specified URL, but the problem is that I am not allowed to use any HTTP libraries, I am only allowed to use TCP.

Below is the code that reads the response message:

private static String readMultiline (BufferedReader inStr) {
    String message="";
    String line=readLine(inStr);

    while (line != null) {
         message += line + "\r\n";
         line=readLine(inStr);
    }

    if (message.length() == 0) return null;
        return message;
}

private static String readLine (BufferedReader inStr) {
    String line = null;
    try{
         line = inStr.readLine();
    } catch (IOException ioe) {
         System.out.println("ERROR: Incoming packet could not be read properly.");
         return null;
    }
    return line;
}

The problem is that the header and the web page content are completely received, but the while loop still waits for the 'next' line, which does not exist. After a while, timeout occurs and the code continues. The server I am trying to connect to does not do "Connection:close". How can I detect the end of the file?

Is this homework? Why would you be banned from using the HTTP libraries that are included in the Java SDK? — Perception, Dec 15 '11 at 22:14
Yes, it is a homework. Although, as far as I can see, HTTP libraries work pretty much the same way when it comes to reading from the stream. As far as it cannot detect the end of the file, the problem will occur. — kubuzetto, Dec 15 '11 at 22:15

score 3 · Accepted Answer · answered Dec 15 '11 at 22:25

3

Other answers were deleted so I will post one up. You are using an IO method that read lines of information, with a line being defined as a set of characters terminated by a newline character. But there is no guarantee that your TCP packets are ending in newline, so you need to use a more agnostic IO read to process them.

Take a look at DataInputStream, it has a convenience method called readFully that I think you will find quite helpful.

answered Dec 15 '11 at 22:25

Perception

79,279
19
185
195

Wow I didn't know about readFully(), that's exactly what I was looking for. Thanks a lot :D – kubuzetto Dec 15 '11 at 22:27

score 0 · Answer 2 · answered Dec 15 '11 at 22:19

0

read up on the HTTP protocol. look for length information.

answered Dec 15 '11 at 22:19

jtahlborn

52,909
5
76
118

Thank you, I will try to parse the header and use BufferedReader.read() instead of readLine(). – kubuzetto Dec 15 '11 at 22:25

score 0 · Answer 3 · answered Jan 31 '12 at 16:16

sorry for the late answer.

For anyone who might have suffered the same problem, I also forgot to add "Connection: close" line to the request itself. When I added it, timeout problem disappeared. With this and read()/readFully() functions, I managed to solve the problem. Hope it helps anyone.

Java HTTP GET response waits until timeout

3 Answers3

Linked