13

I have been reading some articles about TCP implementation on Linux and I got confused, what is the difference between net.ipv4.tcp_max_syn_backlog and net.core.somaxconn and the backlog passed as parameter to listen() system call, and what is the relation between them.

P.S. I want explanation for kernel 4.15 because I found that there are some differences between oldest and newer kernels on this subject.

red0ct
  • 4,840
  • 3
  • 17
  • 44
Random
  • 355
  • 1
  • 3
  • 12

1 Answers1

23

sysctl is an API. So you can just read the Linux kernel documentation for appropriate version:

tcp_max_syn_backlog - INTEGER
    Maximal number of remembered connection requests, which have not
    received an acknowledgment from connecting client.
    The minimal value is 128 for low memory machines, and it will
    increase in proportion to the memory of machine.
    If server suffers from overload, try increasing this number.

somaxconn - INTEGER
    Limit of socket listen() backlog, known in userspace as SOMAXCONN.
    Defaults to 128.  See also tcp_max_syn_backlog for additional tuning
    for TCP sockets.

Let's consider a TCP-handshake.. tcp_max_syn_backlog represents the maximal number of connections in SYN_RECV queue. I.e. when your server received SYN, sent SYN-ACK and haven't received ACK yet. This is a separate queue of so-called "request sockets" - reqsk in code (i.e. not fully-fledged sockets, "request sockets" occupy less memory. In this state we can save some memory and not yet allocate a full socket because the full connection may not be at all in the future if ACK will not arrive). The value of this queue is affected (see this post) by listen()'s backlog argument and limited by tcp_max_syn_backlog in kernel.

somaxconn represents the maximal size of ESTABLISHED queue. This is another queue.
Recall the previously mentioned SYN_RECV queue - your server is waiting for ACK from client. When the ACK arrives the kernel roughly speaking makes the big full-fledged socket from "request socket" and moves it to ESTABLISHED queue. Then you can do accept() on this socket. This queue is also affected by listen()'s backlog argument and limited by somaxconn in kernel.

Useful links: 1, 2.

red0ct
  • 4,840
  • 3
  • 17
  • 44
  • 1
    in the link you mentioned i found this, "The behavior of the backlog argument on TCP sockets changed with Linux 2.2. Now it specifies the queue length for completely established sockets waiting to be accepted, instead of the number of incomplete connection requests. The maximum length of the queue for incomplete sockets can be set using /proc/sys/net/ipv4/tcp_max_syn_backlog.", but you said that the backlog affect both of the queues. can you explain ? – Random Jun 30 '20 at 06:38
  • @Random Did you read [this SO post](https://stackoverflow.com/questions/58183847/does-listens-backlog-number-include-syn-received-connections-count-in-case-of-t/58185850) I attached? It's well explained in my answer (see `sk_max_ack_backlog`). Please read before arguing. – red0ct Jun 30 '20 at 08:27
  • 1
    @Random It is the same in 4.15 kernel: [`sk_acceptq_is_full`](https://elixir.bootlin.com/linux/v4.15.18/source/include/net/sock.h#L833) and [`inet_csk_reqsk_queue_is_full`](https://elixir.bootlin.com/linux/v4.15.18/source/include/net/inet_connection_sock.h#L294) make comparison with `sk->sk_max_ack_backlog` which is set through `listen()` call. – red0ct Jun 30 '20 at 08:35
  • Sorry bro, i'm just trying to understand and i didn't meant anything, so if backlog param is < max_syn_backlog then, backlog param is the max length of syn queue, otherwise the max length of the syn queue will be max_syn_backlog, and the same thing with the accept queue and somaxconn ?, is that right ? – Random Jun 30 '20 at 08:50
  • 1
    @Random Yep. *"The backlog argument in the `listen()` syscall is truncated to the somaxconn value. So, the somaxconn value shouldn't exceed 65535 (USHRT_MAX)"*. Added link. If my answer helped, you can mark it as accepted. – red0ct Jun 30 '20 at 10:20
  • Thank you so much, it helped me a lot, just one thing, if you know some articles or books or docs that explains TCP stack of newer kernels > 4.3, please refer them, because i will do some modification and i need a full understand of the stack, i already read the book TCP/IP ARCHITECTURE, DESIGN, AND IMPLEMENTATION IN LINUX but it's for kernel 2.3 and i didn't found the same architecture in the kernel 4.15 – Random Jun 30 '20 at 10:40
  • 1
    @Random You're welcome :) The Linux kernel is changing very rapidly, so it's hard to keep up with books for newest versions. In general it could be sufficient to understand the fundamentals of network stack on some well-described Linux kernel version and then just inspect the source code (to see the differences, analyze new features added inspecting git Linux kernel repo) for some specific version you need to comprehend. – red0ct Jun 30 '20 at 11:02
  • @Random There is a good book [Linux Kernel Networking: Implementation and Theory, Rami Rosen](https://www.amazon.com/Linux-Kernel-Networking-Implementation-Experts-ebook/dp/B01HXJH7QA) for 4.9 kernel. – red0ct Jun 30 '20 at 11:03
  • 4
    Given how little memory a syn queued connection takes up (i read 80 bytes), what is the downside to bumping this value to something very large. From 512 to to say 50,000. That's still only 4 MB of memory. And would make you significantly more resistant to syn spoof attacks. Is there a downside? – l008com Oct 07 '20 at 13:08