C read all input once optimization

Question

first I'm looking for optimization, fast time execution

I would like to read data from input in C so here is my code (Linux)

int main(void) {
    char command_str[MAX_COMMAND_SIZE];

    while (!feof(stdin)) {
        fgets(command_str, MAX_COMMAND_SIZE, stdin);
        // Parse data
    }
    return EXIT_SUCCESS;
}

According to this post Read a line of input faster than fgets? read() function seems to be the solution.

The data input is like:

100 C
1884231 B
8978456 Z
...

From a file, so I execute my program like ./myapp < mytext.txt

It is not possible to know how many entries there is, it's could be 10, 10000 or even more.

From this post

Drop all the casts on malloc and realloc; they aren't necessary and clutter up the code

So if I use a dynamic array my app will be slower I think.

The idea is:

• Read the whole input in one go into a buffer.

• Process the lines from that buffer.

• That's the fastest possible solution.

If someone would help me. Thanks in advance.

Answer 1

while (!feof(f)) is always wrong. Use this instead:

#include <stdio.h>

int main(void) {
    char command_str[MAX_COMMAND_SIZE];

    while (fgets(command_str, MAX_COMMAND_SIZE, stdin)) {
        // Parse data
    }
    return EXIT_SUCCESS;
}

Reading file contents faster than fgets() is feasible, but seems beyond your skill level. Learn the simple stuff first. There is an awful lot that can be achieved with standard line by line readers... Very few use cases warrant the use of more advanced approaches.

If you want to read the whole input and parse it as a single string, here is a generic solution that should work for all (finite) input types:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

int main(void) {
    size_t pos = 0, size = 1025, nread;
    char *buf0 = malloc(size);
    char *buf = buf0;

    for (;;) {
        if (buf == NULL) {
            fprintf(stderr, "not enough memory for %zu bytes\n", size);
            free(buf0);
            exit(1);
        }
        nread = fread(buf + pos, 1, size - pos - 1, stdin);
        if (nread == 0)
            break;

        pos += nread;
        /* Grow the buffer size exponentially (Fibonacci ratio) */
        if (size - pos < size / 2)
            size += size / 2 + size / 8;
        buf = realloc(buf0 = buf, size);
    }
    buf[pos] = '\0';

    // parse pos bytes of data in buf as a string
    printf("read %zu bytes\n", strlen(buf));        

    free(buf);
    return EXIT_SUCCESS;
}

Answer 2

Maybe you could use fseek (stdin, 0, SEEK_END) to go to the end of the standard input stream, then use ftell (stdin) to get its size in bytes, then allocate memory to save all that in a buffer and then process it's contents.