UNIX socket magic. Recommended for high performance application?

Question

I'm looking using to transfer an accept()ed socket between processes using sendmsg(). In short, I'm trying to build a simple load balancer that can deal with a large number of connections without having to buffer the stream data.

Is this a good idea when dealing with a large number (let's say hundreds) of concurrent TCP connections? If it matters, my system is Gentoo Linux

What matters more than concurrent number of connections is how many _new_ connections will be accepted per second. One? Ten? One hundred? Each could lead to one thousand simultaneous TCP sessions (though it make take different lengths of time to get there). — sarnold, Jun 29 '11 at 10:48
There are hundreds of new connections coming in per second. It follows a typical request/response pattern, the lifetime of each connection is very brief. — Mike, Jun 29 '11 at 10:58

score 3 · Answer 1 · edited May 23 '17 at 12:16

3

You can share the file descriptor as per the previous answer here.

Personally, I've always implemented servers using pre-fork. The parent sets up the listening socket, spawns (pre-forks) children, and each child does a blocking accept. I used pipes for parent <-> child communication.

edited May 23 '17 at 12:16

Community

1
1

answered Jun 29 '11 at 10:47

odrm

5,149
1
18
13

score 1 · Accepted Answer · answered Jun 29 '11 at 10:55

Until someone does a benchmark and establishes how "hard" it is to send a file descriptor, this remains speculation (someone might pop up: "Hey, sending the descriptor like that is dirt-cheap"). But here goes.

You will (likely, read above) be better off if you just use threads. You can have the following workflow:

Start a pool of threads that just wait around for work. Alternatively you can just spawn a new thread when a request arrives (it's cheaper than you think)
Use epoll(7) to wait for traffic (wait for connections + interesting traffic)
When interesting traffic arrives you can just dispatch a "job" to one of the threads.

Now, this does circumvent the whole descriptor sending part. So what's the catch ? The catch is that if one of the threads crashes, the whole process crashes. So it is up to you to benchmark and decide what's best for your server.

Personally I would do it the way I outlined it above. Another point: if the workers are children of the process doing the accept, sending the descriptor is unnecessary.

UNIX socket magic. Recommended for high performance application?

2 Answers2