10

I have apache server running on Ubuntu. Client connects and downloads an image. I need to extract RTT estimations for the underlying TCP connection. Is there a way to do this? Maybe something like running my tcp stack in debug mode to have it log this info somewhere?

Note that I don't want to run tcpdump and extract RTTs from the recorded trace! I need the TCP stack's RTT estimations (apparently this is part of the info you can get with TCP_INFO socket option). Basically need something like tcpprob (kprobe) to insert a hook and record the estimated RTT of the TCP connection on every incoming packet (or on every change).

UPDATE:

I found a solution. rtt, congestion window and more can be logged using tcpprobe. I posted an answer below.

kakhkAtion
  • 2,264
  • 22
  • 23
  • http://stackoverflow.com/questions/16231600/fetching-the-tcp-rtt-in-linux – indiv Jan 20 '16 at 20:58
  • Thanks! Yeah I'm familiar with "ss", but it's not exactly what I need as it will give me a very coarse sampling of the RTT. I need something like tcpprob (kprobe) to insert a hook and record the estimated RTT of the TCP connection on every incoming packet (or on every change). – kakhkAtion Jan 20 '16 at 21:14
  • 2
    Hmm. Well Linux congestion control algorithms are pluggable modules in the `net/ipv4/` directory. Each computes rtt differently (and [reno](http://lxr.free-electrons.com/source/net/ipv4/tcp_cong.c) doesn't compute it at all!), so I'm thinking you'll have to hook into a particular implementation. I see you tagged cubic, so that's [tcp_cubic.c](http://lxr.free-electrons.com/source/net/ipv4/tcp_cubic.c). I don't think there's a generic method of getting the rtt, but I could certainly be wrong... – indiv Jan 20 '16 at 21:55
  • Thanks @indiv ! Yeah I haven't yet been able to find a generic way to do it, I probably need to get my hands dirty and do it myself. I am also experimenting with ss and see if it actually suffice for what I need. I'll keep you posted. – kakhkAtion Jan 20 '16 at 22:19
  • Oh boy! I can't believe tcpprobe was doing this for me the whole time and I missed it! haha I will post an answer in a bit – kakhkAtion Jan 20 '16 at 23:32
  • Have you tried this: https://stackoverflow.com/a/48370480/43046 ? – andresp Feb 02 '19 at 21:41
  • Interesting. No I have not, but will look into it. Thanks! – kakhkAtion Feb 04 '19 at 18:07

2 Answers2

4

This can be done without the need for any additional kernel modules using the ss command (part of the iproute package), which can provide detailed info on open sockets. It won't show it for every packet but most of this info is calculated over a number of packets. E.g. To list the currently open TCP (t option) sockets and associated internal TCP info (i) information - including congestion control algorithm, rtt, cwnd etc:

ss -ti

Here's some example output:

State       Recv-Q Send-Q        Local Address:Port        Peer Address:Port
ESTAB       0      0             192.168.56.102:ssh        192.168.56.1:46327
  cubic wscale:6,7 rto:201 rtt:0.242/0.055 ato:40 mss:1448 rcvmss:1392
advmss:1448 cwnd:10 bytes_acked:33169 bytes_received:6069 segs_out:134
segs_in:214 send 478.7Mbps lastsnd:5 lastrcv:6 lastack:5 
pacing_rate 955.4Mbps rcv_rtt:3 rcv_space:28960
Pierz
  • 7,064
  • 52
  • 59
  • ss documentation does not have any of those fields. E.g. what does "rtt: x/y" even mean? – Alexander Gonchiy Mar 06 '20 at 11:35
  • 1
    For details on the fields take a look at the man page for `ss` linked to in the answer (Some distribution man pages seem to lack this info). For full details see the src code: https://git.kernel.org/pub/scm/network/iproute2/iproute2.git/tree/misc/ss.c#n2468 – Pierz Mar 10 '20 at 14:29
3

This can be done using tcpprobe, which is a module that inserts a hook into the tcp_recv processing path using kprobe records the state of a TCP connection in response to incoming packets.

Let's say you want to probe tcp connection on port 443, you need to do the following:

sudo modprobe tcp_probe port=443 full=1
sudo chmod 444 /proc/net/tcpprobe
cat /proc/net/tcpprobe > /tmp/output.out &
pid=$!

full=1: log on every ack packet received

full=0: log on only condo changes (if you use this your output might be empty)

Now pid is the process which is logging the probe. To stop, simply kill this process:

kill $pid

The format of output.out (according to the source at line 198):

[time][src][dst][length][snd_nxt][snd_una][snd_cwnd][ssthresh][snd_wnd][srtt][rcv_wnd]
kakhkAtion
  • 2,264
  • 22
  • 23