18

I have a C++ multithreaded application which uses posix pipes in order to perform inter thread communications efficiently (so I don't have to get crazy with deadlocks).

I've set the write operation non-blocking, so the writer will get an error if there is not enough space in the buffer to write.

if((pipe(pipe_des)) == -1)
    throw PipeException();

int flags = fcntl(pipe_des[1], F_GETFL, 0); // set write operation non-blocking
assert(flags != -1);
fcntl(pipe_des[1], F_SETFL, flags | O_NONBLOCK);

Now I'd wish to set the pipe buffer size to a custom value (one word in the specific case).

I've googled for it but I was not able to find anything useful. Is there a way (possibly posix compliant) to do it?

Thanks

Lorenzo

PS: I'm under linux (if it may be useful)

Zeruel
  • 183
  • 1
  • 1
  • 5
  • 1
    This is a totally inappropriate use of assert(), unless your program only runs on platforms for which fcntl() never has an error. – William Pursell Mar 07 '11 at 11:03
  • 1
    I think you should just learn how to use synchronization primitives. Using a pipe will increase the overhead by about 100 times, and it seems like it can't achieve what you want anyway. – R.. GitHub STOP HELPING ICE Mar 07 '11 at 18:04
  • I know how to use sync primitives :) Actually I also have a version using sync primitives... Looking to the test results, the version with pipes is at least speeder as the sync one (in some cases pipes are speeder...) – Zeruel Mar 07 '11 at 18:32
  • Instead of pipes, you can use a unix socketpair. `setsockopt(fd, SOL_SOCKET, SO_SNFBUF, size)` is the call you want to set the buffer size. – peterh Jun 15 '16 at 01:50

3 Answers3

20

Since you mentioned you are on Linux and may not mind non-portability, you may be interested in the file descriptor manipulator F_SETPIPE_SZ, available since Linux 2.6.35.

int pipe_sz = fcntl(pipe_des[1], F_SETPIPE_SZ, sizeof(size_t));

You'll find that pipe_sz == getpagesize() after that call, since the buffer cannot be made smaller than the system page size. See fcntl(2).

Jeremy Fishman
  • 1,044
  • 11
  • 11
4

I googled "linux pipe buffer size" and got this as the top link. Basically, the limit is 64Kb and is hard coded.

Edit The link is dead and it was probably wrong anyway. The Linux pipe(7) man page says this:

A pipe has a limited capacity. If the pipe is full, then a write(2) will block or fail, depending on whether the O_NONBLOCK flag is set (see below). Different implementations have different limits for the pipe capacity. Applications should not rely on a particular capacity: an application should be designed so that a reading process consumes data as soon as it is available, so that a writing process does not remain blocked.

In Linux versions before 2.6.11, the capacity of a pipe was the same as the system page size (e.g., 4096 bytes on i386). Since Linux 2.6.11, the pipe capacity is 16 pages (i.e., 65,536 bytes in a system with a page size of 4096 bytes). Since Linux 2.6.35, the default pipe capacity is 16 pages, but the capacity can be queried and set using the fcntl(2) F_GETPIPE_SZ and F_SETPIPE_SZ operations. See fcntl(2) for more information.

Anyway, the following still applies IMO:

I'm not sure why you are trying to set the limit lower, it seems like a strange idea to me. If you want the writer to wait until the reader has processed what it has written, you should use a pipe in the other direction for the reader to send back an ack.

JeremyP
  • 84,577
  • 15
  • 123
  • 161
  • 1
    I am sending tasks through the pipe. What I want is to have an asynchrony degree which is configurable through a parameter passed to the application. I think that it should be an efficient solution to incorporate this behaviour into the pipe without keeping track of additional infos and acks... – Zeruel Mar 07 '11 at 11:14
  • The bottom line is you can't do it the way you want. – JeremyP Mar 07 '11 at 11:44
  • link is dead btw – Volodymyr Boiko Nov 17 '18 at 19:26
2

You could use a shared-memory area( System V - like ) of two words, one for sending data and the other for receiving data, and implement your pipes with them. other solutions, as you may found previously, are about recompiling the kernel as you would like to have it, but it is not the case, I suppose.

Ciao!

iampiergiu
  • 21
  • 2