How to clean a buffer filled using read() and continue reading at the same buffer?

Question

I have a project for the university where I have to implement an Iterative server.

The server uses a protocol given by the professor and in a few words the client has to send a message in a specific form and my server has to make some parsing in order to save some data in a global struct.

I use the function read() in order to receive the message and store it in a char array with fixed size. My problem is that the message some times might be bigger than the size of the buffer I use to store it. NOTE: I am not able to send the size of the message first in order to bypass this problem.

I would like to know if there is a way to make this happen.

Bellow is some of the code:

#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <string.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <unistd.h>
#include "keyvalue.h"
#include <errno.h>

int main ( int argc, char *argv[])
{
    char keyvaluebuffer[4096];
    int num;
    //socket()
    //bind()
    //listen()

    while(1)
       {
           //accept()
           while((num = read(acceptfd, keyvaluebuffer, 4096) ) > 0 )
              {
                  //At this point I use the keyvaluebuffer to parse it and store the data.
              }//while (read > 0) end
           if (num < 0){close(accept);}

           close(accept);
      }//while(1) end       
   close(socket);   
}//main end

Is there some reason not to dynamically allocate `keyvaluebuffer`, reallocating as necessary? — ad absurdum, May 07 '17 at 00:37
@DavidBowling In general I would prefer a solution with the static array because it is something I totally understand, however i would like to know if their is a way to solve this problem using malloc(). — arkas, May 07 '17 at 00:40
What's the question? Why can't you just use a bigger buffer? NB There is no 'static array' here. It's a local variable. — user207421, May 07 '17 at 00:54
@EJP The message might be 100000bytes and I do not think that the size [100000] is the best way to fix this problem — arkas, May 07 '17 at 00:57
Well you need to understand that TCP may only give you one byte at a time. You need to keep reading until you have the entire message, which just means copying out the part you already have and appending the new bit as it arrives. There is no such thing as 'clearing the `read()` buffer'. It's just an array in your own code, you do with it what you like. — user207421, May 07 '17 at 02:28

score 0 · Accepted Answer · answered May 07 '17 at 01:51

If the message is larger than the buffer size, then you will at least need to store the part of the message that has been retrieved before reading more of the message. One option would be to dynamically allocate a message buffer. After reading part of the message into the static buffer it is copied into the dynamic message buffer. If the dynamic message buffer has been filled, it is reallocated.

In the example code below, buffer[] is an array that holds BUF_SIZE chars. msg is a pointer to dynamically allocated memory, and initially points to BUF_SIZE bytes. msgbuff_sz keeps track of the size of the dynamic message allocation, while msg_sz keeps track of the actual message size. After each call to read(), the contents of buffer[] are copied to the appropriate location in msg, and msg_sz is updated. If the message size is the same as the message buffer size, the message buffer msg is reallocated. This continues until the end-of-file condition is reached (or until read() returns an error value).

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <fcntl.h>
#include <unistd.h>

#define BUF_SIZE  4096

int main(void)
{
    int fd = open("message.txt", O_RDONLY);
    if (fd < 0) {
        perror("Unable to open file");
        exit(EXIT_FAILURE);
    }

    char buffer[BUF_SIZE];
    char *msg = malloc(BUF_SIZE);
    size_t msgbuff_sz = BUF_SIZE;
    size_t msg_sz = 0;

    if (msg == NULL) {
        perror("Buffer allocation error");
        exit(EXIT_FAILURE);
    }

    ssize_t ret_val;

    while ((ret_val = read(fd, buffer, sizeof buffer)) > 0) {
        memcpy(msg + msg_sz, buffer, ret_val);
        msg_sz += ret_val;
        if (msg_sz == msgbuff_sz) {
            msgbuff_sz += BUF_SIZE;

            char *tmp = realloc(msg, msgbuff_sz);

            if (tmp == NULL) {
                perror("Buffer reallocation error");
                exit(EXIT_FAILURE);
            }
            msg = tmp;
        }
    }

    if (ret_val < 0) {
        perror("Unable to read file");
        exit(EXIT_FAILURE);
    }

    for (size_t i = 0; i < msg_sz; i++) {
        putchar(msg[i]);
    }
    putchar('\n');

    free(msg);

    return 0;
}

Thank you so much and sorry for the late answer. Your way seems to be working great but there is a small problem. the while(read()>0) never exits. The ret_val never becomes zero. — arkas, May 07 '17 at 14:34
My answer was more concerned with the buffer storage issue, and works when reading from a file. `read()` should return 0 if the other end closes the socket. If not, handling the situation may be a little more complicated. There is some discussion of the issues [here](http://stackoverflow.com/questions/3053757/read-from-socket) and [here](http://stackoverflow.com/questions/666601/what-is-the-correct-way-of-reading-from-a-tcp-socket-in-c-c). — ad absurdum, May 07 '17 at 14:50

How to clean a buffer filled using read() and continue reading at the same buffer?

1 Answers1