Multi-threaded Server handling multiple clients in one thread

Question

I wanted to create a multi-threaded socket server using C++11 and standard linux C-Librarys.

The easiest way doing this would be opening a new thread for each incoming connection, but there must be an other way, because Apache isn't doing this. As far as I know Apache handles more than one connection in a Thread. How to realise such a system?

I thought of creating one thread always listening for new clients and assigning this new client to a thread. But if all threads are excecuting an "select()" currently, having an infinite timeout and none of the already assigned client is doing anything, this could take a while for the client to be useable.

So the "select()" needs a timeout. Setting the timeout to 0.5ms would be nice, but I guess the workload could rise too much, couldn't it?

Can someone of you tell me how you would realise such a system, handling more than one client for each thread? PS: Hope my English is well enough for you to understand what I mean ;)

Answer 1

The standard method to multiplex multiple requests onto a single thread is to use the Reactor pattern. A central object (typically called a SelectServer, SocketServer, or IOService), monitors all the sockets from running requests and issues callbacks when the sockets are ready to continue reading or writing.

As others have stated, rolling your own is probably a bad idea. Handling timeouts, errors, and cross platform compatibility (e.g. epoll for linux, kqueue for bsd, iocp for windows) is tricky. Use boost::asio or libevent for production systems.

Here is a skeleton SelectServer (compiles but not tested) to give you an idea:

#include <sys/select.h>

#include <functional>
#include <map>

class SelectServer {
 public:
  enum ReadyType {
    READABLE = 0,
    WRITABLE = 1
  };

  void CallWhenReady(ReadyType type, int fd, std::function<void()> closure) {
    SocketHolder holder;
    holder.fd = fd;
    holder.type = type;
    holder.closure = closure;
    socket_map_[fd] = holder;
  }

  void Run() {
    fd_set read_fds;
    fd_set write_fds;
    while (1) {
      if (socket_map_.empty()) break;

      int max_fd = -1;
      FD_ZERO(&read_fds);
      FD_ZERO(&write_fds);
      for (const auto& pr : socket_map_) {
        if (pr.second.type == READABLE) {
          FD_SET(pr.second.fd, &read_fds);
        } else {
          FD_SET(pr.second.fd, &write_fds);
        }
        if (pr.second.fd > max_fd) max_fd = pr.second.fd;
      }

      int ret_val = select(max_fd + 1, &read_fds, &write_fds, 0, 0);
      if (ret_val <= 0) {
        // TODO: Handle error.
        break;
      } else {
        for (auto it = socket_map_.begin(); it != socket_map_.end(); ) {
          if (FD_ISSET(it->first, &read_fds) ||
              FD_ISSET(it->first, &write_fds)) {
            it->second.closure();
            socket_map_.erase(it++);
          } else {
            ++it;
          }
        }
      }
    }
  }

 private:
  struct SocketHolder {
    int fd;
    ReadyType type;
    std::function<void()> closure;
  };

  std::map<int, SocketHolder> socket_map_;
};

Answer 2

First off, have a look at using poll() instead of select(): it works better when you have large number of file descriptors used from different threads.

To get threads currently waiting in I/O out of waiting I'm aware of two methods:

1. You can send a suitable signal to the thread using pthread_kill(). The call to poll() fails and errno is set to EINTR.
2. Some systems allow a file descriptor to be obtained from a thread control device. poll()ing the corresponding file descriptor for input succeeds when the thread control device is signalled. See, e.g., Can we obtain a file descriptor for a semaphore or condition variable?.

Answer 3

This is not a trivial task.

In order to achieve that, you need to maintain a list of all opened sockets (the server socket and the sockets to current clients). You then use the select() function to which you can give a list of sockets (file descriptors). With correct parameters, select() will wait until any event happen on one of the sockets.

You then must find the socket(s) which caused select() to exit and process the event(s). For the server socket, it can be a new client. For client sockets, it can be requests, termination notification, etc.

Regarding what you say in your question, I think you are not understanding the select() API very well. It is OK to have concurrent select() calls in different threads, as long as they are not waiting on the same sockets. Then if the clients are not doing anything, it doesn't prevent the server select() from working and accepting new clients.

You only need to give select() a timeout if you want to be able to do things even if clients are not doing anything. For example, you may have a timer to send periodic infos to the clients. You then give select a timeout corresponding to you first timer to expire, and process the expired timer when select() returns (along with any other concurrent events).

I suggest you have a long read of the select manpage.