3

pipe buffer size can be set from c using fcntl(). Can it be done directly from a shell script?

Charles Duffy
  • 280,126
  • 43
  • 390
  • 441
Andreas
  • 5,086
  • 3
  • 16
  • 36
  • 1
    Possible duplicate of [Add a big buffer to a pipe between two commands](https://stackoverflow.com/questions/8554568/add-a-big-buffer-to-a-pipe-between-two-commands) – PaSTE Aug 01 '18 at 15:05
  • 1
    @Andreas, ...could you provide some additional context for the question, so one can better determine whether it *is* in fact a duplicate? – Charles Duffy Aug 01 '18 at 15:30
  • @CharlesDuffy Trying to make a sync pulse with handshake using files simplest way possible, which could be used to control frame rate of an application. Found one way yesterday on SO (can't find it right now, lol) mentioning `stdbuf` to control stdio buffering, and then I deduced that `tail` can be used to bridge the gap from stdio to named pipes; `stdbuf -o0 tail -f my_sync_fifo`. `stdbuf` pretty much does what I want. – Andreas Aug 02 '18 at 07:20
  • That being the case, it sounds like your issue wasn't with content being stuck in the pipe but buffered in libc (which is, indeed, where folks more frequently get surprised). Anyhow, in terms of whether to close the question as a dupe -- while as-presently-asked this question wasn't very applicable to your use case, it *could* be given a distinct and correct answer by someone who was willing to include a bit of C, so I'm tempted to leave it open. – Charles Duffy Aug 02 '18 at 11:11
  • @CharlesDuffy yeah, well, zero is also a size, is it not? :-) would you say `stdbuf` and `tail` combo is an answer to the question? if no, why not? (for the record, my own research indicates the most versatile solution being to write a tiny C program for current use case because making a sync pulse in shell turned out tricky) – Andreas Aug 02 '18 at 14:55
  • @Andreas, turning off stdio buffering on the reader or writer sides (as with `stdbuf`) *doesn't* change the pipe buffer size. They're two different buffers. But the way the pipe buffer works means it *doesn't* change your program's behavior in the way you're trying to avoid -- it always, by nature, flushes output as soon as the reader is prepared to consume it. – Charles Duffy Aug 02 '18 at 14:58
  • You can verify this yourself -- read through https://github.com/coreutils/coreutils/blob/master/src/libstdbuf.c; there's no `F_SETPIPE_SZ` anywhere in there. – Charles Duffy Aug 02 '18 at 15:01
  • @CharlesDuffy wow, what was some bad code. more caveats than features. thanks for the link, I'll be running the other way now (doing my own C hack ) – Andreas Aug 02 '18 at 17:49

1 Answers1

1

As far as I know, there is no standard linux command-line tool that hooks into F_SETPIPE_SZ. You might want to take a look at this previous question which uses buffer to add a user-sized buffer between two commands. While it cannot reduce the effective size of the pipe below its default value, it can increase the effective size of the pipe buffer well beyond the OS limit of /proc/sys/fs/pipe-max-size.

PaSTE
  • 4,050
  • 18
  • 26
  • A standard command-line tool would be tricky to work with, unless you're going to have it adjust and `exec`, so something like `... | with-pipe-size 0 16384 -- some-command` to exec through to `some-command` after running `fcntl(0, F_SETPIPE_SZ, 16384)`. Absent that kind of usage mode, though, this would need to be a shell capability. – Charles Duffy Aug 01 '18 at 15:19