0

Probably a silly question, with read and other functions you can specify the number of bytes you want to read, however when reading from stdin I find that I can only type 1024 characters in the prompt, if I type the 1025 character, it's not written and if I want the line to be read (pressing ENTER key) I need to remove the 1024 character in order to leave space for '\n' I suppose. This occurs only in my c program not the shell so what's causing this restriction?

#include <unistd.h>
#include <stdio.h>

int main() {
    char buf[2048];
    int c;

    c = read(fileno(stdin), &buf, sizeof(buf));
    printf("%s\n", buf);

    return 0;
}
Daniel
  • 139
  • 13
  • 3
    This is a property of the terminal driver on your system, rather than of the program or the C library. The shell doesn't read a single line; it reads characters as they become available. See also [Canonical vs non-canonical terminal input](https://stackoverflow.com/questions/358342/). – Jonathan Leffler Oct 26 '18 at 22:28
  • 2
    Note that `read()` doesn't add a null terminator to the input that it reads, but `printf()` expects a null-terminated string. – Barmar Oct 26 '18 at 22:37
  • 1
    Instead of adding a null terminator, you could tell `printf()` how many characters to print — `printf("%.*s\n", c, buf);`. – Jonathan Leffler Oct 26 '18 at 22:39
  • Thanks for the observation, but that's not the issue here.. I would like to be able to read the 2048 characters specified for _buf_ or at least find out why I can't – Daniel Oct 26 '18 at 22:43
  • Which O/S (operating system) are you running this on? – ChuckCottrill Oct 26 '18 at 23:03
  • 2
    If you use an open source o/s, you can modify the terminal driver source code and recompile your kernel to allow you to type more than 1 KiB on a single line, but anything much short of that isn't going to work. The terminal driver imposes a limit; you have to change the terminal driver to change that limit. If you're on Linux, you can poke around `/proc` file system to see if there's a dynamic configuration parameter you can change (so you don't have to recompile the kernel, but you do have to alter the settings of the terminal driver), but I've not heard of that being possible. – Jonathan Leffler Oct 27 '18 at 00:38
  • 3
    The limit can be a nuisance if you copy'n'paste more than 1 KiB of text with no newlines in it from a browser and want to paste it into a file on your system. Use a program such as Vim to manage it — it puts the terminal into a non-canonical mode and therefore doesn't run into the limit. – Jonathan Leffler Oct 27 '18 at 00:44
  • 1
    @ChuckCottrill I'm on OS X 10.13 – Daniel Oct 27 '18 at 15:13
  • thanks all, I thought I would be something simpler than recompiling the kernel – Daniel Oct 27 '18 at 15:17

1 Answers1

0

Transferring select comments to form an answer.

General diagnosis

This is a property of the terminal driver on your system, rather than of the program or the C library. Modern shells such as Bash don't read a single line; they read characters as they become available using non-canonical input. See also Canonical vs non-canonical terminal input.

Barmar noted:

Note that read() doesn't add a null terminator to the input that it reads, but printf() expects a null-terminated string.

Instead of adding a null terminator, you could tell printf() how many characters to print:

printf("%.*s\n", c, buf);

That is, however, tangential to the question of how to get a long line of input.

If you use an open source o/s, you can modify the terminal driver source code and recompile your kernel to allow you to type more than 1 KiB on a single line, but anything much short of that isn't going to work. The terminal driver imposes a limit; you have to change the terminal driver to change that limit. If you're on Linux, you can poke around the /proc file system to see if there's a dynamic configuration parameter you can change (so you don't have to recompile the kernel, but you do have to alter the settings of the terminal driver); I've not heard of that being possible.

The limit can be a nuisance if you copy'n'paste more than 1 KiB of text with no newlines in it from a browser and want to paste it into a file on your system. Use a program such as Vim to manage it — it puts the terminal into a non-canonical mode and therefore doesn't run into the limit.

Using POSIX termios to slurp input from a terminal

If you want a program to read from a terminal without the line lengths (but also with line editing such as erase or kill processing), then you could consider this program — slurp:

/*
@(#)File:           $RCSfile: slurp.c,v $
@(#)Version:        $Revision: 1.3 $
@(#)Last changed:   $Date: 2018/10/28 17:14:24 $
@(#)Purpose:        Put terminal into non-canonical mode to slurp input
@(#)Author:         J Leffler
*/

/*TABSTOP=4*/

#include "posixver.h"
#include "stderr.h"
#include <assert.h>
#include <fcntl.h>
#include <signal.h>
#include <stdbool.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <termios.h>
#include <unistd.h>

static const char optstr[] = "a:ho:V";
static const char usestr[] = "[-hV][-a output | -o output]";
static const char hlpstr[] =
    "  -a output  Append to named file (creating it if necessary)\n"
    "  -h         Print this help message and exit\n"
    "  -o output  Output to named file (truncating it if it exists)\n"
    "  -V         Print version information and exit\n"
    ;

static struct termios saved = { 0 };
static bool sigint_enabled = false;
static bool sigquit_enabled = false;
static bool slurping = false;

static void reset_termios(void);
static void set_non_canonical(void);
static void sig_handler(int signum);
static void set_signal_handling(void);
static void slurp(int ofd, const char *filename);

#ifndef lint
/* Prevent over-aggressive optimizers from eliminating ID string */
extern const char jlss_id_slurp_c[];
const char jlss_id_slurp_c[] = "@(#)$Id: slurp.c,v 1.3 2018/10/28 17:14:24 jonathanleffler Exp $";
#endif /* lint */

int main(int argc, char **argv)
{
    const char *filename = "standard output";
    int ofd = STDOUT_FILENO;
    int oflag = 0;

    err_setarg0(argv[0]);

    int opt;
    while ((opt = getopt(argc, argv, optstr)) != -1)
    {
        switch (opt)
        {
        case 'h':
            err_help(usestr, hlpstr);
            /*NOTREACHED*/
        case 'o':
        case 'a':
            if (ofd != STDOUT_FILENO)
            {
                err_remark("the -a and -o flags are mutually exclusive\n");
                err_usage(usestr);
            }
            oflag = (opt == 'o') ? O_TRUNC : O_APPEND;
            if ((ofd = open(optarg, O_WRONLY | O_CREAT | oflag, 0644)) < 0)
                err_syserr("failed to open file %s for writing: ", optarg);
            filename = optarg;
            break;
        case 'V':
            err_version("PROG", &"@(#)$Revision: 1.3 $ ($Date: 2018/10/28 17:14:24 $)"[4]);
            /*NOTREACHED*/
        default:
            err_usage(usestr);
            /*NOTREACHED*/
        }
    }

    if (optind != argc)
    {
        err_remark("unexpected file name options (first is '%s')\n", argv[optind]);
        err_usage(usestr);
    }

    set_non_canonical();
    if (slurping)
        set_signal_handling();
    slurp(ofd, filename);

    return 0;
}

static void reset_termios(void)
{
    tcsetattr(STDIN_FILENO, 0, &saved);
}

static void set_non_canonical(void)
{
    if (tcgetattr(STDIN_FILENO, &saved) == 0)
    {
        struct termios modified = saved;
        atexit(reset_termios);
        /*
        ** On macOS 10.14 (at least), if you don't reset ISIG, the
        ** signal characters are not transferred to the program, so
        ** you can't detect those signals.  With ICANON reset, they
        ** don't generate the signal either.  The code does not try
        ** to handle the suspend (^Z) key specially, nor any other
        ** keys than EOF, INTR, QUIT.
        */
        modified.c_lflag &= ~(ICANON | ISIG);
        modified.c_cc[VMIN] = 1;
        modified.c_cc[VTIME] = 0;
        tcsetattr(STDIN_FILENO, TCSANOW, &modified);
        slurping = true;
    }
}

static void sig_handler(int signum)
{
    reset_termios();
    _exit(128 + signum);
}

/* Almost worth a data structure and a loop, but not quite */
static void set_signal_handling(void)
{
    /* Simulate SIGINT and SIGQUIT */
    if (signal(SIGINT, SIG_IGN) != SIG_IGN)
    {
        (void)signal(SIGINT, sig_handler);
        sigint_enabled = true;
    }
    if (signal(SIGQUIT, SIG_IGN) != SIG_IGN)
    {
        (void)signal(SIGQUIT, sig_handler);
        sigquit_enabled = true;
    }
    /* Have program terminate when sent normal signals */
    if (signal(SIGHUP, SIG_IGN) != SIG_IGN)
        (void)signal(SIGHUP, sig_handler);
    if (signal(SIGTERM, SIG_IGN) != SIG_IGN)
        (void)signal(SIGTERM, sig_handler);
    if (signal(SIGPIPE, SIG_IGN) != SIG_IGN)
        (void)signal(SIGPIPE, sig_handler);
}

static void slurp(int ofd, const char *filename)
{
    char buffer[4096];
    int  nbytes;

    while ((nbytes = read(STDIN_FILENO, buffer, sizeof(buffer))) > 0)
    {
        /* Simulate EOF and interrupt and quit signals */
        if (nbytes == 1 && slurping)
        {
            if (buffer[0] == saved.c_cc[VEOF])
                break;
            if (sigint_enabled && buffer[0] == saved.c_cc[VINTR])
                exit(128 + SIGINT);
            if (sigquit_enabled && buffer[0] == saved.c_cc[VQUIT])
                exit(128 + SIGQUIT);
        }
        if (write(ofd, buffer, nbytes) != nbytes)
            err_syserr("failed to write %d bytes to %s: ", nbytes, filename);
    }
}

The library code used is available in my SOQ (Stack Overflow Questions) repository on GitHub as files stderr.c, stderr.h and posixver.h in the libsoq sub-directory.

This deals with most of the traps for the unwary. It does its best to reset the terminal back to the initial ('known good') state when it exits. It does simulate EOF, interrupt and quit keyboard signals, but it does not simulate regular terminal processing such as erase or kill.

It doesn't make sense to use this when the standard input is not a terminal, but the code should handle that OK too (it simply does normal reads). You can send the output to standard output (default) or to a file (-o file to create or truncate a file, -a file to append or create a file).

Jonathan Leffler
  • 730,956
  • 141
  • 904
  • 1,278