How can we limit file descriptor to always be less than 1024?

Question

There is limitation in select system call that it will not work beyond 1024. This is what document says

WARNING: select() can monitor only file descriptors numbers that
       are less than FD_SETSIZE (1024)—an unreasonably low limit for
       many modern applications—and this limitation will not change.
       All modern applications should instead use poll(2) or epoll(7),
       which do not suffer this limitation.

we have experienced this behaviour only in AWS EC2 instance. Outside of AWS fd is always < 1024

Is there a setting in Linux system (Suse Linux 15 SP2) that will always create fds < 1024?

I don't think so, did you consider using `poll` as the warning suggests? Also, why the AWS tag? — David Ranieri, Nov 16 '21 at 15:15
Thanks i know poll is an alternative. Yes we have experienced this behaviour only in AWS EC2 instance. Outside of AWS fd is always < 1024 — Meraj Hussain, Nov 16 '21 at 15:17
_Yes we have experienced this behaviour only in AWS EC2 instance. Outside of AWS fd is always_ You can add that to the question. — David Ranieri, Nov 16 '21 at 15:19
Note the question [Are there any platforms where using structure copy on an `fd_set` (for `select()` or `pselect()`) causes problems?](https://stackoverflow.com/q/2421672/15168) — Jonathan Leffler, Nov 16 '21 at 15:20
How can you force it? What if the low IDs are already taken? The best you can do is to compare it in your program and fail if it is higher. — Eugene Sh., Nov 16 '21 at 15:20

score 1 · Accepted Answer · answered Nov 16 '21 at 15:31

You don't want to do this. Linux always uses the lowest possible file descriptor. So if you get file descriptor 1024 then it means file descriptors 0 up to 1023 were all used already.

If you make Linux only use file descriptors 0-1023, then your program still won't work, because instead of getting file descriptor 1024, you'll get an error saying there aren't any more file descriptors it can use.

You should:

Make sure your program closes file descriptors when it's done with them.
Maybe the reason descriptors 0-1023 were all used is because you forgot to close them. So many sure your program closes them.
If they were real file descriptors that your program was actually using, not just ones you forgot to close, then continue to step 2...
Use poll instead of select, like the document says. It is not a difficult change.
Consider using epoll, which is more efficient than select when you are polling a lot of file descriptors at the same time. However, it is much more complicated.

How can we limit file descriptor to always be less than 1024?

1 Answers1