How to make scanf to read more than 4095 characters given as input?

Question

We have test application in c which takes input using scanf in string format and that string it uses for further processing.

So far everything was working fine, however lately we have condition where need to input more than 4100 bytes and scanf needs to read them however scanf doesn't read more than 4095 from stdin. Simplest form of problematic code is as follows,

#include <stdio.h>
#include <string.h>

int main()
{
    char input_array[5000];
    int len;
    printf("Enter key: ");
    scanf("%s",input_array);
    len = strlen(input_array);

    printf("Message: %s\n",input_array);
    printf("Message Len: %d\n",len);
    return 0;
}

I think this is happening because scanf can max read two lines and one line size is 2k.(Correct me if I am wrong).

Now this code works if we read characters from file but that way we need to change other test code also :(

Currently we are copying and pasting 4200 characters on terminal to give input to scanf.

Now my questions are,
Is there a way to instruct scanf to read more than 2 line?
Is there any other function which we can use which doesn't have this limitation ?

Answer 1

It is because your terminal inputs are buffered in the I/O queue of the kernel.

Input and output queues of a terminal device implement a form of buffering within the kernel independent of the buffering implemented by I/O streams.

The terminal input queue is also sometimes referred to as its typeahead buffer. It holds the characters that have been received from the terminal but not yet read by any process.

The size of the input queue is described by the MAX_INPUT and _POSIX_MAX_INPUT parameters;

By default, your terminal is in Canonical mode.

In canonical mode, all input stays in the queue until a newline character is received, so the terminal input queue can fill up when you type a very long line.

Now to answer your questions:

Is there a way to instruct scanf to read more than 2 line?

That 2 line concept is wrong. Anyways, you can't instruct scanf to read more than 4096 bytes if the maximum size of I/O queue of the terminal is set to 4096 bytes.

Is there any other function which we can use which doesn't have this limitation ?

No you can't even with any other function. It's not a limitation of scanf.

---

EDIT: Found a rather standard way of doing it

We can change the input mode of terminal from canonical mode to non-canonical mode.

To change the input mode we have to use low level terminal interface.

We can do the task as follows:

#include <stdio.h>
#include <string.h>
#include <termios.h> 
#include <unistd.h>

int clear_icanon(void)
{
  struct termios settings;
  int result;
  result = tcgetattr (STDIN_FILENO, &settings);
  if (result < 0)
    {
      perror ("error in tcgetattr");
      return 0;
    }

  settings.c_lflag &= ~ICANON;

  result = tcsetattr (STDIN_FILENO, TCSANOW, &settings);
  if (result < 0)
    {
      perror ("error in tcsetattr");
      return 0;
   }
  return 1;
}

int main()
{
    clear_icanon(); // Changes the input mode of terminal from canonical mode to non canonical mode.

    char input_array[5000];
    int len;
    printf("Enter key: ");
    scanf("%s",input_array);
    len = strlen(input_array);

    printf("Message: %s\n",input_array);
    printf("Message Len: %d\n",len);
    return 0;
}

In case you want to know how to do it from terminal

$ stty -icanon (changes the input mode to non-canonical)
$ stty icanon (changes it back to canonical)

---

Earlier answer was: (This technique is older)

I don't know whether it is the best way or not, but It can be done by changing the terminal mode from cooked (default) to cbreak or to raw mode.

When the terminal is in cbreak mode, It works with single characters at a time, rather than forcing a wait for a whole line and then feeding the line in all at once.

either you can use stty cbreak in terminal before executing the program.

or

To use cbreak mode programatically

First install the curses package by running

$ sudo apt-get install libncurses5-dev

Next edit the program as follows:

#include <stdio.h>
#include <string.h>
#include <curses.h> 

int main()
{
    initscr(); //start curses mode      
    cbreak(); //change the terminal mode to cbreak. Can also use raw();
    endwin(); //end curses mode

    char input_array[5000];
    int len;
    printf("Enter key:");
    scanf("%s",input_array);
    len = strlen(input_array);

    printf("Message:%s\n",input_array);
    printf("Message Len:%d\n",len);
    return 0;
}

Now compile with the -lcurses option

$ gcc 1.c -lcurses

Answer 2

As ARBY correctly stated: the actual problem is the discrepancy in the buffersizes of the LibC and the terminal. If you accept that limitation you are OK.

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

int main()
{
  char input_array[5000];
  size_t len;
  int res;

  printf("BUFSIZ = %d\n", BUFSIZ);

  while ((res = scanf("%4095s", input_array)) == 1) {
    len = strlen(input_array);
    printf("Message Len:%zu\n", len);
  }
  return 0;
}

Output:

$ dd if=<(yes hi|tr -d '\n') bs=4200 count=2 of=longline
$ gcc-4.9 -O3 -g3  -W -Wall -Wextra  -std=c11 checkbuf.c -o checkbuf
$ ./checkbuf < longline
BUFSIZ = 8192
Message Len:4095
Message Len:4095
Message Len:106

EDIT One, not recommended way to concatenate the results involves a wee bit of pointer-juggling:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

int main()
{
  char input_array[10000];
  char *ptoarr;
  size_t len;
  int res;

  printf("BUFSIZ = %d\n", BUFSIZ);

  ptoarr = input_array;

  while ((res = scanf("%4095s", ptoarr)) == 1) {
    len = strlen(ptoarr);
    // TODO check that total length is smaller than or equal to input_array size
    printf("Message Len:%zu\n", len);
    ptoarr += len;
  }
  len = strlen(input_array);
  printf("Message Len:%zu\n", len);
  return 0;
}

But, as I said, not recommended.