Over TCP communication in C, how can you indicate to stop calling read() for a request without knowing any prior information about the data?

Question

I am currently programming a server in C that communicates over sockets using TCP. The client is supposed to send {filename\n} + {file contents\n} and the server will store that information and then send a response on success. However, we can't be sure the client will actually send the proper information in a structured protocol.

Often in simpler situations, we know a specified amount of bytes to be sent before and can wait until that specified number has been reached. In this case, we don't and currently the code looks like this:

//buffer is a resize-able array, 
//assume it's correct + don't worry about memory leakage for now
resizeablearray_t *buffer;

char data[255];
char retry = 1; 
while (retry) {
   int bytes = read(socketfd, data, 255);
   if (bytes <= 0) {
      fprintf(stderr, "Error %s\n", strerror(errno));
      return;
   }
   push(buffer, data, bytes);
}

Therefore, we're given a huge problem: how do we indicate to the read() function in c that we've read in all the information on the prior call and we shouldn't call it again?

The read() function blocks until there are bytes to be read over the server. However, in our model, if we continue to attempt to read due to short counts and there is nothing left in the buffer, we will wait FOREVER.

Any ideas on how to indicate before another read() call, without any prior information on the incoming message and its structure, that we can break from the while loop and stop attempting to read from the socket?

You need to examine the contents of the `data` array and see if you received the terminator. Using `\n` as the terminator for the filename is fine, since filenames don't contain `\n` characters. For the file contents, I would use `\0` as the terminator, assuming the contents are text. For a binary file, you need to send the length, or base64 encode the contents. — user3386109, Nov 29 '18 at 00:52
There _are_ different ways to do this, but what about sending: `filename\n file_size_in_bytes\n file_contents` since the client will/can know the file size before starting? What precludes this in your application? — Craig Estey, Nov 29 '18 at 00:52
suggest using a `select()` statement with a timeout. However, note that if the returned count from `read()` is 0 that means the other end of the connection has hungup.BTW: the returned type from `read()` is NOT `int` but rather: `ssize_t` — user3629249, Nov 29 '18 at 00:53

score 2 · Answer 1 · answered Nov 29 '18 at 00:58

In modern file transfer protocols, there is an indication of the size of the file, or more directly, how many bytes in the payload of the transaction. This allows the other side to know how many bytes will be delivered. If not enough bytes are delivered, the reader needs to keep reading. When all the bytes are delivered, there is no "extra" call to read in the sense you are asking about.

Even if the sender is creating data dynamically, and thus doesn't know how many bytes are going to be sent in total at the beginning of the transaction, it can at least report how many bytes are being delivered in the current send. When it is done, it can send a final report that indicates it is done sending. The receiver reads until it sees this final report.

If you use the recv call instead of read, you will be able to pass in flags to indicate that you do not wish to block waiting forever, and instead want an immediate status of whether there is anything left to read. However, this is an unreliable method to determine if the transfer is complete. The sender could just be slow, and the rest of the file may very well come some time later.

Craig Estey · Answer 2 · 2018-11-29T01:33:19.580

How about creating a [simple/crude] packet protocol between the two:

byte_length(4)|data(byte_length)

When done, sender sends one final packet with byte_length set to 0 (i.e. an EOF marker) to indicate all has been sent.

Here's some rough code:

//buffer is a resize-able array,
//assume it's correct + don't worry about memory leakage for now
resizeablearray_t *buffer;

char data[255];
char retry = 1;

void
receive(void)
{
    uint32_t size;
    int bytes;

    while (retry) {

        bytes = read(socketfd,&size,sizeof(size));
        if (bytes <= 0) {
            fprintf(stderr, "Error %s\n", strerror(errno));
            return;
        }

        // got EOF marker
        if (size == 0)
            break;

        size = ntohl(size);

        bytes = read(socketfd, data, size);
        if (bytes <= 0) {
            fprintf(stderr, "Error %s\n", strerror(errno));
            return;
        }
        push(buffer, data, bytes);
    }
}

void
sender(void)
{
    uint32_t size;
    int bytes;

    while (retry) {

        bytes = read(filefd, data, sizeof(data));
        if (bytes == 0)
            break;

        if (bytes < 0) {
            fprintf(stderr, "Error %s\n", strerror(errno));
            return;
        }

        size = htonl(bytes);
        bytes = write(socketfd, &size, sizeof(size));

        bytes = write(socketfd, data, bytes);
        if (bytes <= 0) {
            fprintf(stderr, "Error %s\n", strerror(errno));
            return;
        }
    }

    // send EOF marker
    size = 0;
    write(socketfd,&size,sizeof(size));
}

Over TCP communication in C, how can you indicate to stop calling read() for a request without knowing any prior information about the data?

2 Answers2

Linked