HTTP Server Not Sending Complete File To WGET, Firefox. Connection reset by peer?

Question

I'm writing an HTTP server, and am having trouble sending larger files. If I grab them with netcat, the output seems perfect. If I use a browser or wget, I only get the complete file sometimes. WGET keeps getting "connection reset by peer" errors, see the output below. Firefox says "the connection was reset".

Here's the relevant procedure that sends data to the client:

int handle_request(int sockfd, struct cached_file content) {
    char buffer[1024]; // FIXME hardcoded arbitrary buffer size for header

    unsigned int sent_bytes;
    unsigned int total = 0;
    unsigned int bytes_left = content.size;

    printf("I have to send %u bytes of content.\n", content.size);

    snprintf(buffer, 1024, "HTTP/1.1 %s\nContent-Type: %s\nContent-Length: %s\n\n", content.statuscode, content.contenttype, content.sizestring);
    sent_bytes = send(sockfd, buffer, strlen(buffer), MSG_MORE);
    printf("I wanted to send %u bytes of header, and I sent %u.\n", strlen(buffer), sent_bytes);

    while (total < bytes_left) {
        sent_bytes = send(sockfd, content.data+total, bytes_left, 0);
        if (sent_bytes == -1) { 
            printf("send() returned -1\n");
            break; 
        }

        total      += sent_bytes;
        bytes_left -= sent_bytes;
    }
    printf("I sent %u bytes of content. I had %u left to send.\n", total, bytes_left);

    if (sent_bytes == -1)
        logprint("socket error!", errno);
}

Here's the output from wget trying to grab the file:

wget --tries 1 http://localhost:8081/image.jpg
--2015-07-01 13:21:42--  http://localhost:8081/image.jpg
Resolving localhost (localhost)... ::1, 127.0.0.1
Connecting to localhost (localhost)|::1|:8081... failed: Connection refused.
Connecting to localhost (localhost)|127.0.0.1|:8081... connected.
HTTP request sent, awaiting response... 200 OK
Length: 700895 (684K) [image/jpeg]
Saving to: ‘image.jpg.10’

image.jpg.10                      53%[===============================>                             ] 363.31K  --.-KB/s   in 0.001s 

2015-07-01 13:21:42 (688 MB/s) - Read error at byte 372031/700895 (Connection reset by peer). Giving up.

wget --tries 1 http://localhost:8081/image.jpg
--2015-07-01 13:21:43--  http://localhost:8081/image.jpg
Resolving localhost (localhost)... ::1, 127.0.0.1
Connecting to localhost (localhost)|::1|:8081... failed: Connection refused.
Connecting to localhost (localhost)|127.0.0.1|:8081... connected.
HTTP request sent, awaiting response... 200 OK
Length: 700895 (684K) [image/jpeg]
Saving to: ‘image.jpg.11’

image.jpg.11                       6%[==>                                                          ]  42.69K  --.-KB/s   in 0s     

2015-07-01 13:21:43 (500 MB/s) - Read error at byte 43711/700895 (Connection reset by peer). Giving up.

Debugging output from the http server:

I have to send 700895 bytes of content.
I wanted to send 65 bytes of header, and I sent 65.
I sent 700895 bytes of content. I had 0 left to send.

I'd appreciate another set of eyes on this! Why is this happening and how can I fix it?

"The connection was reset." If I use the debugging tools, it tells me that a random amount of bytes less than the total size was transferred - exactly like wget. — Matthew Scheffel, Jul 01 '15 at 17:53
have you checked your file system to assure that it is 'ok' to have a foreign server (even though it is on your machine) write to the directory where the file is to be saved. — user3629249, Jul 01 '15 at 18:08
as a recovery step, for wget, could use the '-c' parameter to try to resume the download — user3629249, Jul 01 '15 at 20:29
in the server code, I think the first call to 'send()' should also have its' returned value checked for errors, not just the later calls to 'send()' — user3629249, Jul 01 '15 at 20:31
Which server, like apache2, are you running on your machine? — user3629249, Jul 01 '15 at 20:33

score 1 · Accepted Answer · answered Jul 01 '15 at 18:18

My guess is that the error is related to code you have not shown here.

It is a common mistake in simple implementation to not fully read the request but instead only read the first line or some bytes to determine the requested page, then send the response and finally close.

Since at the time of the close there are still unread data from the client, this will result in Connection reset by peer. You don't see this effect with nc because the request you send with nc is probably shorter than the request from the browser and thus all data from the request are read in case of nc, but not in case of browser.

Apart from that your response is invalid even if browsers accept it. Your status line (the first line) stops after the status code instead of adding the reason phrase, see http://www.w3.org/Protocols/rfc2616/rfc2616-sec6.html#sec6.1. Also, you use \n instead of \r\n as line delimiter.

I believe this is the exact problem - I am doing the hacky request parsing you mention. The reason phrase is actually in content.statuscode, my naming is misleading here. Thank you! — Matthew Scheffel, Jul 01 '15 at 19:00

score 0 · Answer 2 · answered Jul 01 '15 at 18:04

0

I don't know if it is necessary, but when I inspect the headers being sent by a simple website, there are quotes around the values:

Accept-Ranges:"bytes"
Cache-Control:"max-age=25920000"
Connection:"Keep-Alive"
Content-Length:"4777"
Content-Type:"image/gif"
Date:"Wed, 01 Jul 2015 17:55:52 GMT"
Expires:"Tue, 26 Apr 2016 17:55:52 GMT"
Keep-Alive:"timeout=30, max=100"
Last-Modified:"Mon, 28 Apr 1997 01:25:47 GMT"
Server:"Apache/1.3.28 (Unix) mod_gzip/1.3.19.1a mod_perl/1.28"

Is it possible the content type is not correct?

answered Jul 01 '15 at 18:04

wallyk

56,922
16
83
148

You are mistaken: `echo "GET /w/delorie-soft.gif HTTP/1.0" | netcat delorie.com 80 | head -n 15 HTTP/1.1 200 OK Date: Wed, 01 Jul 2015 18:09:22 GMT Server: Apache/1.3.28 (Unix) mod_gzip/1.3.19.1a mod_perl/1.28 Cache-Control: max-age=25920000 Expires: Tue, 26 Apr 2016 18:09:22 GMT Last-Modified: Mon, 28 Apr 1997 01:25:47 GMT ETag: "e40cd-12a9-3363fc9b" Accept-Ranges: bytes Content-Length: 4777 Connection: close Content-Type: image/gif` – Matthew Scheffel Jul 01 '15 at 18:09
@MatthewScheffel: Hmmm, that means the Firefox Developer pane is misrepresenting the data. That is surprising. – wallyk Jul 01 '15 at 18:23

HTTP Server Not Sending Complete File To WGET, Firefox. Connection reset by peer?

2 Answers2

Linked