2

The book "advanced programming in unix environment" discussed pipe in chapter 15, it shows that we should pay attention to the buffering type when deal with standard I/O functions.

The buffering types for different opened standard I/O streams are (discussed in chapter 5 of the book):

  • standard error is unbuffered
  • streams connected to terminal devices is line-buffered
  • all other streams are fully-buffered

When parent/child connect to a pipe, the end (which should be a FILE * type object, according to the interface) that they used to communicate should be fully-buffered according to the rule list above (since it's a stream connected to pipe). But the behavior of sample code from that chapter seems to be something NOT fully-buffered.

Here is the sample code:

myuclc.c:

1   #include "apue.h"
2   #include <ctype.h>

3   int
4   main(void)
5   {
6       int     c;

7       while ((c = getchar()) != EOF) {
8           if (isupper(c))
9               c = tolower(c);
10          if (putchar(c) == EOF)
11              err_sys("output error");
12          if (c == '\n')
13              fflush(stdout);
14      }
15      exit(0);
16  }

popen1.c:

1   #include "apue.h"
2   #include <sys/wait.h>

3   int
4   main(void)
5   {
6       char    line[MAXLINE];
7       FILE    *fpin;
8
9       if ((fpin = popen("myuclc", "r")) == NULL)  // "myuclc" is executable file compile-link by "myuclc.c"
10          err_sys("popen error");
11      for ( ; ; ) {
12          fputs("prompt> ", stdout);
13          fflush(stdout);
14
15          if (fgets(line, MAXLINE, fpin) == NULL) /* read from pipe */
16              break;
17          if (fputs(line, stdout) == EOF)
18              err_sys("fputs error to pipe");
19      }
20      if (pclose(fpin) == -1)
21          err_sys("pclose error");
22      putchar('\n');
23      exit(0);
24  }

So my question is : fgets() in line 15 of popen1.c should be fully-buffered according to the buffering rules, why does it act like line-buffered or unbuffered:

Furthermore, I also tried to setvbuf() before fgets() to specifically set buffering type to _IOFBF (fully-buffered) of fpin, still not work.

prompt> abc
abc
prompt> ABC
abc
prompt> efg
efg
prompt> EFG
efg
prompt>
Bin
  • 980
  • 2
  • 12
  • 24

2 Answers2

3

In myuclc.c you perform an explicit flush on every newline:

12          if (c == '\n')
13              fflush(stdout);

This causes the stream to be manually line-buffered. Whenever you flush the pipe, the process on the other end will be unblocked and will read whatever was in the buffer at the time.

The "buffering rules" talk about when this flushing happens automatically. Unbuffered streams are automatically flushed after every write command (fprintf, fputc, etc.). Line-buffered streams are automatically flushed whenever a newline character is written to the stream.

And all streams are flushed when the buffer fills up, when the stream is closed, or when the writer performs an explicit flush

jforberg
  • 6,537
  • 3
  • 29
  • 47
  • 1
    Or when the program exits cleanly (via return from main or a call to `exit`). – Shachar Shemesh Sep 02 '16 at 15:30
  • Shachar: yes, I see this as a special case of EOF. – jforberg Sep 02 '16 at 15:31
  • 1
    What do you mean by "when they encounter end-of-file"? That makes little sense for a stream that is being written. – William Pursell Sep 02 '16 at 15:54
  • @jforberg , I'm a little confused after reading your answer and the one by @Kaz . (1) When deal with pipe via standard I/O library, will both sides of pipe has buffering ? It seems `fflush()` can make `fgets()` of the other side immediately get the data, the buffering type it has (which is fully-buffered) doesn't make any effect. (2) I tried to change the `fflush()` to `setvbuf()` in `myuclc.c`, it works, but not work when replace `fflush()` in `myuclc.c` with a `setvbuf()` in the `popen1.c` (before `fgets()`). (3) Any difference of buffering behavior if `pipe` is not involved in our topic ? – Bin Sep 03 '16 at 11:53
  • @WilliamPursell: Yes, it would be better to say "when the stream is closed". Edited. – jforberg Sep 04 '16 at 15:51
  • @Bin: I don't think I understand your question, in any case it would be better to ask a new question on that topic if you desire a more specific answer. – jforberg Sep 04 '16 at 16:02
  • @jforberg, thanks for your reply, let me make it short: In `myuclc.c`, I found it's ok to replace `fflush()` with a call of `setvbuf()` to make the output line-buffered; but not work if remove `fflush()` in `myuclc.c`, and add a `setvbuf()` in `popen1.c` ( yes, I'm trying to set the buffering type of `fgets()` in `popen1.c` ), can you explain it ? many thanks! – Bin Sep 04 '16 at 16:56
  • 1
    @Bin Ok, I think understand. The reader of a pipe has no control over the buffering on the write end, it can only put a limit on the number of bytes it is willing to receive in each read() call. The writer has complete control over when he wants to write and how much he writes before flushing (absent buffer limits of the pipe itself). – jforberg Sep 07 '16 at 13:00
0

Your code doesn't match your description. You're talking about the pipe system call, but the code uses popen. popen is a function not in ISO C but in POSIX, and is subject to its own set of requirements in POSIX. POSIX doesn't say what is the buffering mode of popen-ed streams, unfortunately. It has this curious wording, though: "Buffered reading before opening an input filter may leave the standard input of that filter mispositioned. Similar problems with an output filter may be prevented by careful buffer flushing; for example, with fflush." I cannot make sense of the first sentence: how can reading take place before opening? The second sentence seems to imply that popen streams may be fully buffered, and so explicit fflush may be necessary to be sure data is passed on to an output pipe. Of course, if that process itself is reading input with full buffering, it might not help!

If you create a pipe with the pipe system call, obtaining a pair of file descriptors, you can then create FILE * streams over these descriptors with fdopen. That, again, is not an ISO C function. Therefore it is not bound by the requirement which ISO C gives for fopen, namely: "When opened, a stream is fully buffered if and only if it can be determined not to refer to an interactive device. The error and end-of-file indicators for the stream are cleared." To see whether that is true of fdopen, we have to look into POSIX. Unfortunately, POSIX is silent about this; it doesn't say anything about buffering. It also doesn't say that fdopen inherits any special requirements from fopen. It does say that the meaning of the mode flags is "exactly as specified in fopen(), except that modes beginning with w shall not cause truncation of the file."

POSIX has a description of fopen, and that description mirrors the above quoted ISO C text about buffering, verbatim. Since POSIX's description of fdopen doesn't have any such text and doesn't have any requirement that fdopen must follow requirements from fopen (other than with regard to the meaning of the mode flags), the buffering set up by fdopen is up in the air. A conforming fdopen could set up full buffering even if the file descriptor is a TTY.

Thus, if you're using fdopen or popen, and the choice of buffering matters in your situation, you should arrange it yourself with setvbuf.

Regarding:

I also tried to setvbuf() before fgets() ...

Buffering affects output. The stdio input functions do not delay the delivery of buffered input data to the application. The fact that you're able to read individual lines from the process connected to the pipe means that that process is flushing its own output buffer for each line. That line is then transmitted through the pipe and available to your own process. The stdio library is not going to delay your fgets operation until more lines accumulate, even under full buffering. That's not how it works; full buffering means output is accumulated until a buffer fills up or fflush is called.

Kaz
  • 55,781
  • 9
  • 100
  • 149