Sockets "hanging up" in C/buffer smaller than usual

Question

When I send() and recv() data from my program locally it works fine.

However, on my remote server, the same program, which usually receives data in chunks of 4096, will receive in buffers capped at 1428, which rarely jump above this number.

Worse of all, after a minute or so of transferring data the socket just freezes and stops execution, and the program perpetually stays in this frozen state, like so:

Received: 4096
Received: 4096
Received: 3416

The server is simple, it accepts a connection from a client and receives data in chunks of 4096, which works absolutely fine locally, but on my remote server it is failing consistently, unless I only send a small chunk of data (sending 1000 byte files worked fine).

int main()
{

    while(1){

        int servSock = socket(AF_INET, SOCK_STREAM | SOCK_NONBLOCK, IPPROTO_TCP);
        if(servSock < 0){
            fprintf(stderr, "Socket error.\n");
            continue;
        }

        struct sockaddr_in servAddr;
        memset(&servAddr, 0, sizeof(servAddr));
        servAddr.sin_family = AF_INET;
        servAddr.sin_addr.s_addr = htonl(INADDR_ANY);
        servAddr.sin_port = htons(atoi(STANDARD_PORT));

        if(bind(servSock, (struct sockaddr*) &servAddr, sizeof(servAddr)) < 0){
            fprintf(stderr, "Bind error.\n");
            close(servSock);
            continue;
        }

        if(listen(servSock, BACKLOG) < 0){
            fprintf(stderr, "Listen error.\n");
            close(servSock);
            continue;
        }
        printf("%s", "Listening on socket for incoming connections.\n");

        struct sockaddr_in clntAddr;
        socklen_t clntAddrLen = sizeof(clntAddr);

        while(1) {
        int newsock = accept(servSock, (struct sockaddr*) &clntAddr, &clntAddrLen);
        if(newsock < 0){
            fprintf(stderr, "Accept connection error");
                                return 1;

            continue;
        }

        char clntName[INET_ADDRSTRLEN];
        if (inet_ntop(AF_INET, &clntAddr.sin_addr.s_addr, clntName, sizeof(clntName)) != NULL)
            printf("Handling client %s:%d\n", clntName, ntohs(clntAddr.sin_port));

        char file[17];
        memset(file, 0, 17);
        int recvd =  recv(newsock, file, 16, 0);
        file[17] = '\0';
        char local_file_path[200];
        memset(local_file_path, 0, 200);

        strcat(local_file_path, "/home/");
        strcat(local_file_path, file);
        printf(local_file_path);
        FILE* fp = fopen(local_file_path, "wb");

        char buffer[4096];
        while(1)
        {
            memset(buffer, 0, 4096);
            recvd =  recv(newsock, buffer, 4096, 0);
            printf("Received: %d\n", recvd);
            fwrite(buffer, sizeof(char), recvd, fp);
            if(recvd == -1 || recvd == 0) {
                fclose(fp);
                break;
            } else
        }
        close(newsock);
        }
        close(servSock);
    }
    return 1;
}

EDIT: For more context, this is a Windows server I am adapting to linux. Perhaps the recv() call is blocking when it shouldn't be, I'm going to test with flags.

Answer 1

However, on my remote server, the same program, which usually receives data in chunks of 4096, will receive in buffers capped at 1428, which rarely jump above this number.

Insufficient context has been presented for confidence, but that looks like a plausible difference between a socket whose peer is on the same machine (one connected to localhost, for example) and one whose peer is physically separated from it by an ethernet network. The 1428 is pretty close to the typical MTU for such a network, and you have to allow space for protocol headers.

Additionally, you might be seeing that one system coallesces the payloads from multiple transport-layer packets more or differently than the other does, for any of a variety of reasons.

In any case, at the userspace level, the difference in transfer sizes for a stream socket is not semantically meaningful. In particular, you cannot rely upon one end of the connection to read data in the same size chunks that the other sends it. Nor can you necessarily rely on receiving data in full-buffer units, regardless of the total amount being transferred or the progress of the transfer.

Worse of all, after a minute or so of transferring data the socket just freezes and stops execution, and the program perpetually stays in this frozen state, like so:

"Worst" suggests other "bad", which you have not described. But yes, your code is susceptible to freezing. You will not see EOF on the socket until the remote peer closes their side, cleanly. The closure part is what EOF means for a network socket. The cleanness part is required, at the protocol level, for the local side to recognize the closure. If the other end holds the connection open but doesn't write anything else to it then just such a freeze will occur. If the other side is abruptly terminated, or physically or logically cut off from the network without a chance to close their socket, then just such a freeze will occur.

And indeed, you remarked in comments that ...

Both the client and the server are hanging. The client program just stops sending data, and the server freezes as well.

If the client hangs mid-transfer, then, following from the above, there is every reason to expect that the server will freeze, too. Thus, it sounds like you may be troubleshooting the wrong component.

Perhaps the recv() call is blocking when it shouldn't be, I'm going to test with flags.

There is every reason to think the recv() call is indeed blocking when you don't expect it to do. It's highly unlikely that it is blocking when it shouldn't.

It is possible to set timeouts for socket operations, so that they eventually will fail instead of hanging indefinitely when the remote side fails. Doing so would allow your server to recover, but it would not resolve the client-side issue. You'll need to look into that more deeply.^*

---

^*You might see the client unfreeze after the server times out and closes the connection on its end. Don't take that as a resolution.