stderr and stdout - not buffered vs. buffered?

Question

I was writing a small program that had various console output strings depending upon different events. As I was looking up the best way to send these messages I came across something that was a bit confusing.

I have read that stderr is used to shoot messages directly to the console - not buffered. While, in contrast, I read that stdout is buffered and is typically used to redirect messages to various streams?, that may or may not be error messages, to an output file or some other medium.

What is the difference when something is said to be buffered and not buffered? It made sense when I was reading that the message is shot directly to the output and is not buffered .. but at the same time I realized that I was not entirely sure what it meant to be buffered.

score 3 · Answer 1 · answered Nov 14 '13 at 23:31

3

Typically, stdout is line buffered, meaning that characters sent to stdout "stack up" until a newline character arrives, at which point that are all outputted.

answered Nov 14 '13 at 23:31

Konrad Lindenbach

4,911
1
26
28

Ah, so the characters to be printed to the screen are buffered until the new line character? What if, after the first one is received, no second `new line` character exists? Will the buffered output just be copied to the console all at once? – sherrellbc Nov 14 '13 at 23:37
As some have already pointed out, a newline isn't the only thing that can cause a buffered output to be "flushed". Another potential cause is a certain threshold of number of characters or by the user calling `fflush()` – Konrad Lindenbach Nov 14 '13 at 23:39

score 2 · Answer 2 · answered Nov 14 '13 at 23:34

A buffered stream is one in which you keep writing until a certain threshold. This threshold could be a specific character, as Konrad mentions for line buffering, or another threshold, such as a specific count of characters written.

Buffering is intended to speed up input/output operations. One of the slowest things a computer does is write to a stream (whether a console or a file). When things don't need to be immediately seen, it saves time to store it up for a while.

You are right, stderr is typically an unbuffered stream while stdout typically is buffered. So there can be times when you output things to stdout then output to stderr and stderr appears first on the console. If you want to make stdout behave similarly, you would have to flush it after every write.

score 2 · Answer 3 · answered Nov 14 '13 at 23:38

2

When an output stream is buffered, it means that the stream doesn't necessarily output data the moment you tell it to. There can be significant overhead per IO operation, so lots and lots of little IO operations can create a bottleneck. By buffering IO operations and then flushing many at once, this overhead is reduced.

While stdout and stderr may behave differently regarding buffering, that is generally not the deciding factor between them and shouldn't be relied on. If you absolutely need the output immediately, always manually flush the stream.

answered Nov 14 '13 at 23:38

Max

21,123
5
49
71

Are you referring to flushing the stdout? What is the deciding factor between using either of the streams? – sherrellbc Nov 14 '13 at 23:45
`stdout` is for "output" and `stderr` is for "errors". Here's a good explanation of when to use each http://www.jstorimer.com/blogs/workingwithcode/7766119-when-to-use-stderr-instead-of-stdout – Max Nov 15 '13 at 00:18

ensc · Answer 4 · 2013-11-15T00:25:59.863

1

Assume

int main(void)
{
    printf("foo\n");
    sleep(10);
    printf("bar\n");
}

when executing it on the console

$ ./a.out

you will see foo line and 10 seconds later bar line (--> line buffered). When redirecting the output into a file or a pipe

$ ./a.out > /tmp/file

the file stays empty (--> buffered) until the program terminates (--> implicit fflush() at exit).

When lines above do not contain a \n, you won't see anything on the console either until program terminates.

Internally, printf() adds a character to a buffer. To make things more easy, let me describe fputs(char const *s, FILE *f) instead of. FILE might be defined as

struct FILE {
    int fd;   /* is 0 for stdin, 1 for stdout, 2 for stderr (usually) */
    enum buffer_mode mode;
    char buf[4096];
    size_t count; /* number of chars in buf[] */
};
typedef struct FILE *FILE;

int fflush(FILE *f)
{
    write(f->fd, f->buf, f->count);
    f->count = 0;
}

int fputc(int c, FILE *f)
{
    if (f->count >= ARRAY_SIZE(f->buf))
        fflush(f);
    f->buf[f->count++] = c;
} 

int fputs(char const *s, FILE *f)
{
    while (*s) {
        char c = *s++;

        fputc(c, f);

        if (f->mode == LINE_BUFFERED && c == '\n')
            fflush(f);
    }

    if (f->mode == UNBUFFERED)
        fflush(f);
}

edited Nov 15 '13 at 00:25

answered Nov 14 '13 at 23:34

ensc

6,704
14
22

I did read that all output streams are flushed at exit, so it makes sense. But why is the output to a file buffered until program exit to begin with? What if the output to be written to the file is extremely lengthy? – sherrellbc Nov 14 '13 at 23:40
printf() uses an internal buffer of a certain size (e.g. 4KiB). When this buffer is full, `fflush()` will be called implicitly and output appears. – ensc Nov 14 '13 at 23:43
Intuitively that makes sense but also is a bit confusing. If you were to use something like (a single statement) `printf("printing")`, would fflush() be called immediately before the function returns? When would be a case when `printf()` has it's buffer filled completely and it implicitly flushed? Is it when there are many repeated calls to the `printf()` function ? – sherrellbc Nov 14 '13 at 23:48
Thanks for the code. It makes much better sense now. I am a bit confusing on what you did in the last function `fputs`, though. I follow until you OR a boolean with the `*s`. What is the purpose of this code and the code that follows it? – sherrellbc Nov 15 '13 at 00:22

stderr and stdout - not buffered vs. buffered?

4 Answers4

Linked