0

Possible Duplicate:
Pipe buffer size is 4k or 64k?

  1. In linux, which header file specifies the size available for writes on a pipe ?

  2. I capture latency of my main application per configurable cycle and write that data to a pipe. A separate reporting process reads off that pipe. Typically, the main application exchanges about 10,000 messages per second. So, given a cycle of one second, the main application collects 10k latency data points for each message exchange and then writes them off to a pipe on a second's boundary. I have following questions in this scenario

    • Is there way to specify the size of pipe while creation,so i can ensure there is adequate write space in the pipe?
    • Are writes to pipe expensive? How is pipe implemented? Do writes to pipe go against some mmap file or in-memory buffer?
Community
  • 1
  • 1
Jimm
  • 8,165
  • 16
  • 69
  • 118
  • 4
    See http://stackoverflow.com/questions/4624071/pipe-buffer-size-is-4k-or-64k?rq=1 for the pipe buffer size – John Carter Dec 16 '12 at 21:09
  • What you write to a pipe goes into a kernel-side buffer which is discarded once one side of the pipe closes. On Linux, pipes usually have 64k buffers. – fuz Dec 16 '12 at 21:40
  • Your Q2: there's no way to configure the pipe that I know of (no standard way, as in POSIX-provided, for sure). Writes to pipes (and reads from them) are designed to be fast and inexpensive; they are a crucial IPC mechanism in Unix and must be fast. The memory used for piped data is normally in the kernel buffer pool; it does not usually hit disk at all. (It might hit disk if a pipe is full but the process that should be reading it is ignoring the pipe for long enough and there is enough pressure on the kernel buffer pool.) – Jonathan Leffler Dec 16 '12 at 21:43
  • I'm not sure this question should be closed. There are a lot of questions here not covered in the other answer. Perhaps more importantly, the correct answer has changed in the intervening two years as kernel version 2.6.35 changed the pipe size and made it programmable via fcntl(2). – DigitalRoss Dec 16 '12 at 23:08
  • 1
    just become one out 3 questions are answered by below link, it does not make this post an exact duplicate. In fact,the unanswered questions form the bulk of this post. http://stackoverflow.com/questions/4624071/pipe-buffer-size-is-4k-or-64k – Jimm Dec 17 '12 at 03:34

1 Answers1

1
  • Is there way to specify the size of a pipe at creation? Maybe. Since Linux 2.6.35 you can use fcntl(2) with the F_SETPIPE_SZ operation to set the pipe buffer up to /proc/sys/fs/pipe-max-size. On earlier versions, no, but I suppose you could use the socket mechanism instead. It would be slower for most purposes but you can specify the amount of buffering up to wmem_max, see socket(7), and you have certain other controls over kernel memory allocation.
  • Are writes to pipe expensive? No. But write(2) is a kernel call, so pipe I/O should be buffered if possible.
  • How are pipes implemented? With kernel code that transfers data in and out of the system buffer cache.
  • Do writes to pipe go against some mmap file or in-memory buffer? It's an in-memory buffer
DigitalRoss
  • 143,651
  • 25
  • 248
  • 329