Increasing limit of FD_SETSIZE and select

Question

I want to increase FD_SETSIZE macro value for my system. Is there any way to increase FD_SETSIZE so select will not fail

Answer 1

Per the standards, there is no way to increase FD_SETSIZE. Some programs and libraries (libevent comes to mind) try to work around this by allocating additional space for the fd_set object and passing values larger than FD_SETSIZE to the FD_* macros, but this is a very bad idea since robust implementations may perform bounds-checking on the argument and abort if it's out of range.

I have an alternate solution that should always work (even though it's not required to by the standards). Instead of a single fd_set object, allocate an array of them large enough to hold the max fd you'll need, then use FD_SET(fd%FD_SETSIZE, &fds_array[fd/FD_SETSIZE]) etc. to access the set.

Answer 2

I also suggest using poll if possible. And there exist several "event" processing libraries like libevent or libev (or the event abilities of Glib from GTK, or QtCore, etc) which should help you. There are also things like epoll. And your problem is related to C10k

Answer 3

It would be better (and easy) to replace with poll. Generally poll() is a simple drop-in replacement for select() and isn't limited by the 1024 of FD_SETSIZE...

fd_set fd_read;
int id = 42;
FD_ZERO(fd_read);
FD_SET(id, &fd_read);
struct timeval tv;
tv.tv_sec = 5;
tv.tv_usec = 0;
if (select(id + 1, &fd_read, NULL, NULL, &tv) != 1) {
   // Error.
}

becomes:

struct pollfd pfd_read;
int id = 42;
int timeout = 5000;
pfd_read.fd = id;
pfd_read.events = POLLIN;
if (poll(&pfd_read, 1, timeout) != 1) {
   // Error
}

You need to include poll.h for the pollfd structure.

If you need to write as well as read then set the events flag as POLLIN | POLLOUT.

Answer 4

In order to use a fd_set larger than FD_SETSIZE, it is possible to define an extended one like this :

#include <sys/select.h>
#include <stdio.h>

#define EXT_FD_SETSIZE 2048
typedef struct
{
    long __fds_bits[EXT_FD_SETSIZE / 8 / sizeof(long)];
} ext_fd_set;

int main()
{
    ext_fd_set fd;
    int s;
    printf("FD_SETSIZE:%d sizeof(fd):%ld\n", EXT_FD_SETSIZE, sizeof(fd));
    FD_ZERO(&fd);
    while ( ((s=dup(0)) != -1) && (s < EXT_FD_SETSIZE) )
    {
        FD_SET(s, &fd);
    }
    printf("select:%d\n", select(EXT_FD_SETSIZE,(fd_set*)&fd, NULL, NULL, NULL));
    return 0;
}

This prints :

FD_SETSIZE:2048 sizeof(fd):256

select:2045

---

In order to open more than 1024 filedescriptors, it is needed to increase the limit using for instance ulimit -n 2048.

Answer 5

Actually there IS a way to increase FD_SETSIZE on Windows. It's defined in winsock.h and per Microsoft themselves you can increase it by simply defining it BEFORE you include winsock.h:

See Maximum Number of Sockets an Application Can Use (old link), or the more recent page Maximum Number of Sockets Supported.

I do it all the time and have had no problems. The largest value I have used was around 5000 for a server I was developing.