C Sockets: recv() blocks when all data is downloaded

Question

I'm writing a wrapper for Berkley sockets on Windows and Linux. The test program got problem here:

char buf[BUFSIZE];
int res = 0;
while((res = NetRecv(sock, buf, BUFSIZE, 0)) > 0) // 'NetRecv' is pointing to 'recv'
{
    buf[res-1] = '\0';
    printf("%s", buf);
}

The response is to a HTTP-Get request of a web-page content. The socket is streaming.

The 'NetRecv' is initialized correctly - that is, no type mismatch of the functions' pointers there is, I've checked it.

So, Windows version works flawlessly, the Linux one is stuck after reading all page. Namely, the previous to the last 'NetRecv' call accepts last chunk of the response, outputs it, and the next (last) call just blocks. Closing the terminal causes 'SIGHUP' signal. Looks like the Linux version just doesn't realize, that it received the last chunk of data and waits for more.

Is it as it should be? Don't understand then, for what reason there is blocking call possibility. Now, I surely could make non-blocking call and use 'select', but do I really have to?

Thanks in advance)

EDIT: Minimal working example (all checks are omitted and net functions are the standard ones, which also were tested):

int sock = socket(AF_INET, SOCK_STREAM, 0);

// Here getting good IP address of google.com - no problem here
char serv_ip[IPADDR_BUFSIZE];
GetHostAddrByName(AF_INET, "www.google.com", serv_ip, IPADDR_BUFSIZE);
//                 ip ver        site        out buf   out buf size
// The routine above is made with 'getaddrinfo', to be precise

printf("Current IP of '%s' is '%s'.\n", SERV_URL, serv_ip);

// Copying IP string to address struct
struct sockaddr_in addr;
NetIpFromStr(AF_INET, serv_ip, &addr.sin_addr);
addr.sin_family = AF_INET;
addr.sin_port = NetHtons(80);

connect(sock, (const struct sockaddr*)&addr, sizeof(addr));

const char* msg = "GET / HTTP/1.1\r\n\r\n";
send(sock, msg, strlen(msg), 0);

char buf[BUFSIZE];
int res = 0;
while((res = recv(sock, buf, BUFSIZE-1, 0)) > 0)
{
    buf[res] = '\0';
    printf("%s", buf);
}

EDIT 2: Important notice: the Windows version also blocks the call, when all the data is read. Closing the terminal just doesn't crash the program, like it happens in Linux. Therefore, the whole question is such: How to realize that all data is read?

Please provide [minimal complete and verifiable example](https://stackoverflow.com/help/mcve) to increase your chances of an answer. — Inrin, Aug 19 '18 at 11:41
buf[res-1] = '\0'; overwrites the last char received. Use 'buf[res] = '\0';' and oversize the buffer by one; 'char buf[1+BUFSIZE];', or read short: 'NetRecv(sock, buf, BUFSIZE-1, 0)'. — Martin James, Aug 19 '18 at 11:48
Which version of HTTP? If the connection is not closed by the server, and you have not parsed the header to get the content-length, you will not know when you have received all your data. — Martin James, Aug 19 '18 at 11:51
Enen better - don't rely on the NUL terminator at all in network code:) — Martin James, Aug 19 '18 at 11:55
@MartinJames, thanks for arithmetic issue, didn't noticed. Not the key to the topic problem though. The rest you've asked see in the edit. Didn't get about NULL terminator. To what do you refer? I'm setting '\0' in the loop just to output the string to a terminal — master_clown, Aug 19 '18 at 13:09
This is not a minimal working example. Far from it. A minimal working exsmple needs to be working. Yours isn't, as it lacks necessary include directives and function definitions. To check whether your example is working, copy it in its entirety, paste it into an online compiler if your choice, and hit the big button. — n. m. could be an AI, Aug 19 '18 at 13:23

score 2 · Answer 1 · edited Oct 07 '21 at 11:04

The problem is that you are blindly reading from the socket in a loop until an error occurs. Once you have received the entire response, you go back to the socket and keep reading, which then blocks because there is nothing left to read. The only error that can occur at this point is when the connection is closed (or lost), which the server is likely not doing since you are sending an HTTP 1.1 request, where keep-alive is the default behavior for 1.1 (see RFC 2616 Section 8.1 Persistent Connections)

The correct solution is to parse the HTTP response and stop reading from the socket when you reach the end of the response, NOT simply relying on the server to close the socket. Read RFC 2616 Section 4.4 Message Length for how to detect when you have reached the end of the response. DO NOT read more than the response indicates! Once you stop reading, then you can decide whether to close your end of the socket, or reuse it for a new request.

Have a look at this pseudo code for the type of parsing and reading logic you need to use.

Also, your HTTP request is malformed, as you are not sending a required Host header, so no matter what, you will always receive a 400 Bad Request response from any HTTP 1.1 compliant server:

const char* msg = "GET / HTTP/1.1\r\n"
                  "Host: www.google.com\r\n" // <-- add this!
                  "\r\n";

score -1 · Answer 2 · answered Aug 19 '18 at 14:39

The solution was to shutdown the socket for reading, both in Windows and Linux:

// after sending a request:
shutdown(sock, SD_SEND);     // or 'SHUT_WR' in Linux

// now read loop

Curiously, 'shutdown' was called in Winsock tutorials too, but I thought that was unnecessary.

C Sockets: recv() blocks when all data is downloaded

2 Answers2

Related