0

In my application, I have one process (the "writer") reading data from external hardware. This process should supply consistent packets to several "readers".

Question: How can one process send consistent data-packets (not blocking) to several clients while they see something like the end of a personal fifo? I'm on Debian Linux.

1) In my first approach I tried "datagram - unix domain sockets", which worked well. But with the "writer" as server, all the clients have to poll the server permanently. :-( They get one packet several times; or miss a packet if not polling fast enough.

2) My second approach was FIFOs (Named Pipes) which works too, but with several readers "strange things happen", what I got confirmed here: http://beej.us/guide/bgipc/output/html/multipage/fifos.html

I tried this, searched the net and stackoverflow the whole day, but I could not find a reasonable answer.

Edit: Sorry, I did not mention: I can't use socketpair() and fork. My programs are independently developed. I'd like to have the writer ready, while developing new readers.

jww
  • 97,681
  • 90
  • 411
  • 885
Vinz
  • 25
  • 1
  • 6
  • Show your code, in particular your event loop.... I don't grasp what you mean by *IPC-concept*. – Basile Starynkevitch Jan 02 '14 at 19:36
  • a simple example of the `mkfifo()`-approach with loops you can find here[1]. But this does not work with serveral readers, so it's the wrong _IPC-concept_. [1] http://beej.us/guide/bgipc/output/html/multipage/fifos.html – Vinz Jan 02 '14 at 19:49
  • Show *exactly* the loop around `poll(2)` (or whatever event loop you have implemented). Even with a single reader and several writer channels, you need such a loop (you need it as soon as you deal with more than one file descriptor). – Basile Starynkevitch Jan 02 '14 at 19:50
  • The right "IPC-concept" when dealing with several file descriptors (either reading, writing, or both, or connecting) is an **event loop**. So please show your code implementing it! – Basile Starynkevitch Jan 02 '14 at 20:00
  • A simple example-code for the _Datagram Unix Domain Sockets_ you can find at the bottom of: http://www.thomasstover.com/uds.html If you include timeouts for `recvfrom()` and `switch() case`the `bytes_received` of this example, you can handle serveral "events" inside the loop. – Vinz Jan 02 '14 at 20:02
  • I don't see any event loop (with a multiplexing syscall like `poll`) in all the links you mention, and I notice they only handle one single file descriptor, which is not the question you are asking. With several file descriptors you need an event loop (or some multi-threading approach, which is not relevant for your issues). So please **show the code of your event loop** .... – Basile Starynkevitch Jan 02 '14 at 20:04
  • What do you mean with "dealing with several file descriptors"? The __writer__ does not know, how many __readers__ are connected. Should it? The __readers__ do have only one channel for receiving information. I can't **show you much more code** than the examples, mine is nearly the same. – Vinz Jan 02 '14 at 20:08
  • Each channel (socket, pipe, fifo, ...) from some reader to the writer is (at least one) file descriptor. The writer needs to know all of them. So you have to deal with *many* file descriptors *at once*. Hence you need some loop around a multiplexing syscall (`poll`). So please show it, and **show _exactly_ your real code** for us to understand it. – Basile Starynkevitch Jan 02 '14 at 20:12
  • Please believe me, my code is nearly the same as the examples. There is no multiplexing syscall and no many file descriptors. Isn’t my Question nearly the same as this: http://stackoverflow.com/questions/9982046/one-to-many-ipc – Vinz Jan 02 '14 at 20:20
  • ZeroMQ (mentioned in [this](http://stackoverflow.com/a/9982464/841108) answer) or MPI (mentioned in [that](http://stackoverflow.com/a/9982284/841108) one) has its event loop. You need one (calling `poll` or the obsolete `select`). Show it (or use an existing one, and tell us which). And we need your *exact* code (not some which is "nearly the same") to help you! – Basile Starynkevitch Jan 02 '14 at 20:25
  • 1
    Thank you very much for your effort, I'll work with it and come back with a concrete code. Isn't there something like __IP Multicast Datagrams__ for `AF_UNIX` sockets? To get one send to many. – Vinz Jan 02 '14 at 20:35

2 Answers2

2

If the writer process is a server, it might fork the client processes and just plain pipe(2) for communication. If there is no parent/child relationship, consider named pipes made with mkfifo(3), or AF_UNIX sockets (see unix(7) and scoket(2) ....) which are bidirectional (AF_UNIX sockets are much faster than TCP/IP or UDP/IP on the same machine).

Notice that your writer process is reading data from your hardware device and is writing or sending data to several reader clients. So your writer process deals with many file descriptors simultaneously (the hardware device to be read, and the sockets or pipes to be written to the clients, at least one file descriptor per client).

However, the important thing is to have some event loop (notably on the server side, and probably also inside clients). This means that you call some multiplexing syscall like poll(2) in the loop, and you "decide" if you are reading or writing or connecting (and which file descriptor should be read, or should be written, or should connect) in each iteration. See also read(2), write(2), connect(2), send(2), recv(2) etc... Notice that you should buffer data with an event loop (since read and write can be on "partial" or "incomplete" messages).

Notice that poll is not eating CPU resources when waiting for I/O. You could, but you should not anymore, use some older multiplexing syscall (like the obsolete select(2) ...). Use poll(2).

You may want to use a library to provide the event loop, e.g. libevent or libev... See also this answer. That event loop should also (on the server side) poll then read the hardware device.

If some of the programs are using a GUI toolkit (e.g. on the client side) like Qt or Gtk they should profit of the existing event loop provided by that toolkit...

You should read Advanced Linux Programming and know about the C10K problem.

If signals or timers are important (read carefully signal(7) and time(7)) the Linux specific signalfd(2) and timerfd_create(2) could be very helpful since they play nicely with event loops. These linux specific syscalls (signalfd & timerfd_create ...) are too recent to be mentioned in Advanced Linux Programming.

BTW, you could study the source code of existing free software similar to yours, and/or use strace(1) to understand the exact syscalls that they are doing.

If you have no loop around a multiplexing syscall (à la poll(2)) then you have no event loop and your design is buggy and cannot possibly and reliably work (since you need to react to several file descriptors at once).

You could also use a multi-threaded approach, but it is much more complex and not worth the effort in your particular case.

Community
  • 1
  • 1
Basile Starynkevitch
  • 223,805
  • 18
  • 296
  • 547
  • You *should* have an event loop, so you *should use a multiplexing syscall* like `poll(2)`. Your link don't mention that! And with an event loop you *are* able to handle several readers and several writers (actually, many of them). – Basile Starynkevitch Jan 02 '14 at 18:16
  • In my first approach I use _datagram sockets_ `AF_UNIX` with `socket(AF_UNIX, SOCK_DGRAM, 0)`. In my second approach I use _named pipes_ with `mknod(FIFO_NAME, S_IFIFO | 0666, 0);` but they can not several readers, see link in question. If I understand your answer, that's what you recommend. I'd like not to poll, but get an event (connect) or simple compare fifo-indexes. The event-loop should no be the problem. Please wait a moment, I have to assimilate your information. ;-) – Vinz Jan 02 '14 at 18:19
  • You really need an event loop. And that means that you need to buffer... Also, please take time to follow all the web links I gave you... – Basile Starynkevitch Jan 02 '14 at 18:19
  • OK, you are absolutely right with the _event-loop_; but in both of my approaches, I have some kind of event-loop. Otherwise e.g. the server would not handle several connections and at same time collect external data. – Vinz Jan 02 '14 at 19:22
  • 2
    The point is the **IPC-concept** I choose. Both approaches I tested (the same you mentioned) are not satisfying, because the generate more problems than they solve. I think one writer, several local IPC readers is a common problem. There should be a common approach. – Vinz Jan 02 '14 at 19:23
0

ZeroMQ has pattern for this problem. Is fast and supports a lot of programming languages. Pub-Sub. See: https://zeromq.org/ (free and open source).

Daniel Bişar
  • 2,663
  • 7
  • 32
  • 54