How many socket connections possible?

Question

Has anyone an idea how many tcp-socket connections are possible on a modern standard Linux server?

(There is in general less traffic on each connection, but all the connections have to be up all the time.)

Answer 1

I achieved 1600k concurrent idle socket connections, and at the same time 57k req/s on a Linux desktop (16G RAM, I7 2600 CPU). It's a single thread http server written in C with epoll. Source code is on github, a blog here.

Edit:

I did 600k concurrent HTTP connections (client & server) on both the same computer, with JAVA/Clojure . detail info post, HN discussion: http://news.ycombinator.com/item?id=5127251

The cost of a connection(with epoll):

• application need some RAM per connection
• TCP buffer 2 * 4k ~ 10k, or more
• epoll need some memory for a file descriptor, from epoll(7)

Each registered file descriptor costs roughly 90 bytes on a 32-bit kernel, and roughly 160 bytes on a 64-bit kernel.

Answer 2

This depends not only on the operating system in question, but also on configuration, potentially real-time configuration.

For Linux:

cat /proc/sys/fs/file-max

will show the current maximum number of file descriptors total allowed to be opened simultaneously. Check out http://www.cs.uwaterloo.ca/~brecht/servers/openfiles.html

Answer 3

A limit on the number of open sockets is configurable in the /proc file system

cat /proc/sys/fs/file-max

Max for incoming connections in the OS defined by integer limits.

Linux itself allows billions of open sockets.

To use the sockets you need an application listening, e.g. a web server, and that will use a certain amount of RAM per socket.

RAM and CPU will introduce the real limits. (modern 2017, think millions not billions)

1 millions is possible, not easy. Expect to use X Gigabytes of RAM to manage 1 million sockets.

Outgoing TCP connections are limited by port numbers ~65000 per IP. You can have multiple IP addresses, but not unlimited IP addresses. This is a limit in TCP not Linux.

Answer 4

10,000? 70,000? is that all :)

FreeBSD is probably the server you want, Here's a little blog post about tuning it to handle 100,000 connections, its has had some interesting features like zero-copy sockets for some time now, along with kqueue to act as a completion port mechanism.

Solaris can handle 100,000 connections back in the last century!. They say linux would be better

The best description I've come across is this presentation/paper on writing a scalable webserver. He's not afraid to say it like it is :)

Same for software: the cretins on the application layer forced great innovations on the OS layer. Because Lotus Notes keeps one TCP connection per client open, IBM contributed major optimizations for the ”one process, 100.000 open connections” case to Linux

And the O(1) scheduler was originally created to score well on some irrelevant Java benchmark. The bottom line is that this bloat benefits all of us.

Answer 5

On Linux you should be looking at using epoll for async I/O. It might also be worth fine-tuning socket-buffers to not waste too much kernel space per connection.

I would guess that you should be able to reach 100k connections on a reasonable machine.

Answer 6

depends on the application. if there is only a few packages from each client, 100K is very easy for linux. A engineer of my team had done a test years ago, the result shows : when there is no package from client after connection established, linux epoll can watch 400k fd for readablity at cpu usage level under 50%.

Answer 7

Which operating system?

For windows machines, if you're writing a server to scale well, and therefore using I/O Completion Ports and async I/O, then the main limitation is the amount of non-paged pool that you're using for each active connection. This translates directly into a limit based on the amount of memory that your machine has installed (non-paged pool is a finite, fixed size amount that is based on the total memory installed).

For connections that don't see much traffic you can reduce make them more efficient by posting 'zero byte reads' which don't use non-paged pool and don't affect the locked pages limit (another potentially limited resource that may prevent you having lots of socket connections open).

Apart from that, well, you will need to profile but I've managed to get more than 70,000 concurrent connections on a modestly specified (760MB memory) server; see here http://www.lenholgate.com/blog/2005/11/windows-tcpip-server-performance.html for more details.

Obviously if you're using a less efficient architecture such as 'thread per connection' or 'select' then you should expect to achieve less impressive figures; but, IMHO, there's simply no reason to select such architectures for windows socket servers.

Edit: see here http://blogs.technet.com/markrussinovich/archive/2009/03/26/3211216.aspx; the way that the amount of non-paged pool is calculated has changed in Vista and Server 2008 and there's now much more available.

Answer 8

Realistically for an application, more then 4000-5000 open sockets on a single machine becomes impractical. Just checking for activity on all the sockets and managing them starts to become a performance issue - especially in real-time environments.