I wonder how MSS is set in SYN packet? Is it a fixed value in one operating system or the value could be changed in the same operating system? I know that the value is different in different operating systems. Also is the MSS value in SYN related to hardware configuration? Thanks.
2 Answers
Whilst the value of MSS in SYN and SYNACK packets are set by the initiator and responder side, respectively, a widely used practice known as MSS clamping can result in the MSS being altered by a network element on the path - this is often used to reduce the MSS of all connections going over some sort of tunnelled link. For example PPPoE is commonly used on residential broadband and requires an MTU of 1492 and corresponding IPv4 MSS of 1452 so whilst the SYN may leave your machine with an MSS of 1460 (assuming you're using Ethernet with an MTU of 1500) but once it passes the MSS clamping ISP router the MSS in SYN packet will subsequently be changed to 1452, and likewise for the incoming/responder's SYNACK packet so the connection proceeds with reduced MSS of 1452. This practice seems to be used instead of Path MTU Discovery which relies upon the use of ICMP Fragmentation Needed responses from the network as these can be lost on poorly configured networks and by certain load balancing techniques.

- 7,064
- 52
- 59
RFC 879 describes how MSS is used and specified.
In short, MSS is specified during TCP handshake via SYN packet. However, this value can later be changed by OS itself or by setting a protocol option.
You can set option TCP_MAXSEG via setsockopt.

- 1,974
- 18
- 22
-
Thanks for your answer. In current existing operating systems (e.g., Windows, Linux), how is MSS set in SYN packet? In one operating system, is MSS set to be a fix value or not in its SYN packet? – ginny Nov 30 '11 at 23:30
-
MSS is set as an option in the SYN packet. In various OS value for the MSS is set differently. For instance in Windows it can be set globally via system registry: http://www.psc.edu/networking/projects/tcptune/OStune/winxp/winxp_stepbystep.html look for TcpWindowSize parameter. On Linux this parameter can be set per connection via TCP_MAXSEG parameter (http://linux.die.net/man/7/tcp) – Vadym Stetsiak Dec 01 '11 at 17:31
-
Thanks for your answer. Right now, I am doing OS detection using TCP fingerprint. It analyses some fields in the SYN packet from remote OSs. So from your answer, values of all field in SYN packets are all fixed in one OS (They have noting to do with the hardware?). They will not change unless later set, right? – ginny Dec 01 '11 at 18:38
-
MSS parameter is fixed per TCP connection. So basically you can fingerprint TCP connection. However, you must not assume that MSS value for all TCP connections will be same. – Vadym Stetsiak Dec 01 '11 at 18:54
-
You mean that in one OS, MSS value could change automatically even no one changes it manually? – ginny Dec 01 '11 at 19:00
-
I mean that if MSS _can_ change it is bad idea to rely on it for TCP packet fingerprinting. – Vadym Stetsiak Dec 01 '11 at 19:12
-
What do you mean 'MSS can change'? Do you mean change automatically or manually? What about other fields in SYN? Are they have the same situation as MSS? – ginny Dec 01 '11 at 19:21
-
MSS can change either way: have a look at this Cisco docs (http://www.cisco.com/en/US/docs/ios/12_2t/12_2t4/feature/guide/ft_admss.html). This article (http://en.wikipedia.org/wiki/TCP/IP_stack_fingerprinting) answers all your questions. – Vadym Stetsiak Dec 01 '11 at 20:40