C stdin limitation is blocking me from reading a line with over 1200 characters

Question

I have this simple program:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

size_t strlen(const char *str)
{
    const char *s;
    for (s = str; *s; ++s);
    return(s - str);
}

/*
 * Initial size of the read buffer
 */
#define DEFAULT_BUFFER 1024

/*
 * Standard boolean type definition
 */
typedef enum { false = 0, true = 1 } bool;

/*
 * Flags errors in pointer returning functions
 */
bool has_err = false;

/*
 * Reads the next line of text from file and returns it.
 * The line must be free()d afterwards.
 *
 * This function will segfault on binary data.
 */
char *readline(FILE *file){
    char *buffer   = NULL;
    char *tmp_buf  = NULL;
    bool line_read = false;
    int  iteration = 0;
    int  offset    = 0;

    if(file == NULL){
        fprintf(stderr, "readLine: NULL file pointer passed!\n");
        has_err = true;

        return NULL;
    }

    while(!line_read){
        if((tmp_buf = malloc(DEFAULT_BUFFER)) == NULL){
            fprintf(stderr, "readLine: Unable to allocate temporary buffer!\n");
            if(buffer != NULL)
                free(buffer);
            has_err = true;

            return NULL;
        }

        if(fgets(tmp_buf, DEFAULT_BUFFER, file) == NULL){
            free(tmp_buf);

            break;
        }

        if(tmp_buf[strlen(tmp_buf) - 1] == '\n') /* we have an end of line */
            line_read = true;

        offset = DEFAULT_BUFFER * (iteration + 1);

        if((buffer = realloc(buffer, offset)) == NULL){
            fprintf(stderr, "readLine: Unable to reallocate buffer!\n");
            free(tmp_buf);
            has_err = true;

            return NULL;
        }

        offset = DEFAULT_BUFFER * iteration - iteration;

        if(memcpy(buffer + offset, tmp_buf, DEFAULT_BUFFER) == NULL){
            fprintf(stderr, "readLine: Cannot copy to buffer\n");
            free(tmp_buf);
            if(buffer != NULL)
                free(buffer);
            has_err = true;

            return NULL;
        }

        free(tmp_buf);
        iteration++;
    }

    return buffer;
}

int main (int argc, char *argv[])
{
    int rows = 0, cols = 0;
    char *line = readline(stdin);

    printf("%s", line);

    return 0;
}

However when I try entering more than 1200 characters in to console, the program stops working within this line of code: fgets(tmp_buf, DEFAULT_BUFFER, file)

The question: How to make C read from stdin without imposing any internal limitations?

P.S. Niether the program works if I try piping the content I need to enter into program with console pipe operators (i.e. <, ./myprogram <input.txt)

Thank you in advance!

Why are you doing that anyway? `malloc()` and `free()` in a loop, to allocate always the same size? what a bad idea is that!!! — Iharob Al Asimi, Mar 08 '15 at 15:52
Why do you have your own version of `strlen` even though you've included `string.h`? — Spikatrix, Mar 08 '15 at 15:59
You have a memory leak. When `readline()` exits normally at the bottom, `*buffer` remains allocated. When you next call `readline()` you again set `buffer = NULL` and so it becomes impossible to `free()` the previously allocated memory (although `buffer` won't "remember" the previous pointer value anyway). — Weather Vane, Mar 08 '15 at 16:01
Thanks guys, the piece of code was obtained from one of the questions on stackoverflow, also the strlen issue is not really relevant it is used to show the problem, on my machine Xamarin wasn't recognising this func so I've added one, the code is not relevant there will not be any human being who will ever get into buffer overflow problems with this app since it was designed for private purposes, thanks anyway for noticing problems! — Lu4, Mar 08 '15 at 18:35
the copyright statement means we cannot (legally) copy the code, so debugging is only available by visually looking at the source. Please do not copyright code, especially if you want anyone else to debug it for you. — user3629249, Mar 11 '15 at 15:13
I was lacking your comment today, it really made my day ))) Removed copyright, now you are allowed to use any debugging tools you like, thanks for fun — Lu4, Mar 11 '15 at 23:45

score 4 · Accepted Answer · edited May 23 '17 at 11:46

4

The problem is not your program; it is the terminal driver. There is an upper limit on the number of unread characters that the terminal driver will hold. It expects lines to be shorter than some limit, which might be 1024 or 1200 or some other value. If you try to type more characters, they simply won't be accepted.

You can try pressing Control-D (on Unix; Control-Z on Windows) to send the pending line from the terminal driver, and then continuing with more characters. That should work, but it is a nuisance to have to keep counting to 1199 (or whatever number is one less than the limit) and then hit Control-D. It is doubly a nuisance if the data was copy'n'pasted from somewhere and therefore you don't have the option of hitting Control-D at appropriate intervals.

You can also try using a non-canonical input mode. That requires care — you need to restore the canonical mode before your program exits if you set non-canonical mode within the program. (Non-canonical mode is what is used by programs such as vim.)

edited May 23 '17 at 11:46

Community

1
1

answered Mar 08 '15 at 16:05

Jonathan Leffler

730,956
141
904
1,278

Ok, I'll be using files then, this is strange by the way, .net applications do not have such limitation afaik – Lu4 Mar 08 '15 at 16:14
@Lu4: There are several possibilities. Most likely (to my not very well-informed mind — I don't use Windows very often) is that .NET code uses the analog of 'non-canonical input' most or all of the time, so the problem doesn't arise. AFAICR, for regular command-line C programs, there was a limit even on Windows for how much data you can enter on a single line. – Jonathan Leffler Mar 08 '15 at 16:16
L: "*.net applications do not have such limitation*" J:"*... is the terminal driver.*" – alk Mar 08 '15 at 16:51

score 0 · Answer 2 · answered Mar 08 '15 at 16:00

I don't know what you did in your code, but this is what your question title is about

#include <stdio.h>
#include <stdlib.h>

char *readline(FILE *file)
 {
    char  *line;
    size_t length;
    size_t count;
    int    chr;

    length = 100;
    line   = malloc(1 + length);
    if (line == NULL)
     {
        fprintf(stderr, "memory exhausted!\n");
        return NULL;
     }
    count = 0;
    while (((chr = fgetc(file)) != EOF) && (chr != '\n'))
     {
        if (count >= length)
         {
            void *pointer;
            length += length;
            pointer = realloc(line, 1 + length);
            if (pointer == NULL)
             {
                fprintf(stderr, "memory exhausted!\n");
                free(line);
                return NULL;
             }
            line = pointer;
         }
        line[count] = chr;
        count      += 1;
     }
    line[count] = '\0';

    return line;
 }

int main(void)
 {
    char *line = readline(stdin);
    if (line != NULL)
        printf("%s\n", line);
    free(line);
    return 0;
 }

The only limitation in this code is memory, and the code will not fail even if there isn't enough memory because that is taken care of as you can see.

It doesn't work, I'll try to make a screencast and show what the problem is, it looks like the terminal is blocking my ability to enter more than 1200 values — Lu4, Mar 08 '15 at 16:05

C stdin limitation is blocking me from reading a line with over 1200 characters

2 Answers2