10

Linux kernels >= 3.9 allow sharing of sockets between processes with in-kernel load-balancing by setting SO_REUSEPORT: http://lwn.net/Articles/542629/

How can this be used for sockets of type AF_UNIX?

It seems, it only works with TCP, not Unix domain sockets.

Here is a Python test program:

import os
import socket

if not hasattr(socket, 'SO_REUSEPORT'):
   socket.SO_REUSEPORT = 15

if True:
   # using TCP sockets
   # works. test with: "echo data | nc localhost 8888"
   s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
   s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT, 1)
   s.bind(('', 8888))
else:
   # using Unix domain sockets
   # does NOT work. test with: "echo data | nc -U /tmp/socket1"
   s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
   s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT, 1)
   try:
      os.unlink("/tmp/socket1")
   except:
      pass
   s.bind("/tmp/socket1")

s.listen(1)
while True:
   conn, addr = s.accept()
   print('Connected to {}'.format(os.getpid()))
   data = conn.recv(1024)
   conn.send(data)
   conn.close()

Start 2 instances, and test by running the following multiple times:

  • echo data | nc localhost 8888 for TCP
  • echo data | nc -U /tmp/socket1 for Unix domain sockets

When using TCP, the incoming clients will get balanced to both servers. With Unix domain sockets, the incoming clients all get connected to the last started server.

oberstet
  • 21,353
  • 10
  • 64
  • 97
  • First sentence of the article you refer: `One of the features merged in the 3.9 development cycle was TCP and UDP support for the SO_REUSEPORT socket option`. It seems to be supported only for TCP and UDP. – Alex Shkop May 27 '14 at 12:23
  • I don't have an exact answer but it looks like kernel have no means of load balancing UNIX sockets. This has a lot of useful information: http://stackoverflow.com/questions/14388706/socket-options-so-reuseaddr-and-so-reuseport-how-do-they-differ-do-they-mean-t – Alex Shkop May 27 '14 at 12:35
  • @AlexShkop Thanks! FWIW, I can set the option on UDS, and that will allow multiple processes to open the same UDS (listening). When the option is not set, it doesn't allow to do so. However, it does not load-balance. I am looking for an authorative answer that either supports the observed behavior or shows how to do LB-balanced shared UDS. – oberstet May 27 '14 at 12:47

2 Answers2

9

This specific kernel patch is documented here:

http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=c617f398edd4db2b8567a28e899a88f8f574798d

As you can see from the list of patched files, the patch only affected the net/ipv4 and net/ipv6 sockets. Unix domain sockets are implemented in net/unix. So, the answer is: no, SO_REUSEPORT will not work with sockets of type AF_UNIX.

youfu
  • 1,547
  • 3
  • 15
  • 21
Hans Then
  • 10,935
  • 3
  • 32
  • 51
2

A small patch, that adds support for SO_REUSEPORT on UNIX sockets was posted, but has been rejected. This patch however did not implement load-balancing over multiple sockets, it only caused bind() not to fail, if the socket file already exists.

This use case was deemed

a really weird corner case from a user's perspective

So there is still the possibility, that a different patch, that would implement load-balancing for UNIX sockets via SO_REUSEPORT, would get accepted.

Sebastian Schrader
  • 1,453
  • 15
  • 19
  • Thanks for the pointers! I still think the use case is valid and significant: having a secure, performant, low overhead IPC mechanism that builds on established user APIs (sockets on UDS). Of course there are other IPC mechanisms (netmap, DDPK) nowerdays .. but those are often more complex compared to the decades old Posix socket API and UDS. – oberstet May 04 '18 at 09:29