getsockopt(...,SO_ORIGINAL_DST,...) occasionally returns client address

Question

I am working on a project that accepts HTTP requests and forwards them onto a destination. We use Linux (2.6.35.14-106.fc14.x86_64) and TPROXY. I'll put in the details below.

The problem I am seeing is that OCCASIONALLY (1 time out of a 1000 sometimes 1 time out of a million), Linux is returning the peer address as the destination address.

Has anyone seen this occur? I saw one note on the web from 2007 so I think that might be a little dated.

I have the following code (pardon the inconsistent methodology displayed here):

struct sockaddr clientaddr;
socklen_t clientlen = sizeof(clientaddr);
int status = getpeername(acceptedSocket, &clientaddr, &clientlen);

char clientName[256];
clientName[0] = '\0';
int clientport = 0;
if (status == 0) {
    inet_ntop(AF_INET, (void *) &((struct sockaddr_in *)&clientaddr)->sin_addr, clientName, 256);
    clientport = ntohs(((struct sockaddr_in *)& clientaddr)->sin_port);
    **printf("Socket::acceptConnection: getpeername :  %s:%d\n", clientName, clientport); fflush(stdout);**
}
else
{
    LOGINFO(WARNING(352), "Socket::acceptConnection: Could not get client from accepted socket.\n");
}

status = getsockopt(acceptedSocket, SOL_IP, SO_ORIGINAL_DST, (struct sockaddr *) &destaddr, &destlen);

if (status == 0) {
    inet_ntop(AF_INET, (void *) &destaddr.sin_addr, destinationName, 256);
    int portnumber = ntohs(destaddr.sin_port);
    ssize_t dl = strlen(destinationName);
    sprintf(&destinationName[dl], ":%d", portnumber);
    **printf("Socket::acceptConnection: getsockopt :  %s\n", destinationName); fflush(stdout);**
}
else
{
    LOGINFO(WARNING(352), "Socket::acceptConnection: Could not get destination from accepted socket.\n");
}

What happens is that most of the time the getpeername and getsockopt report correctly (see the IPTABLE config below).

Unfortunately occasionally get getsockopt report is the same as getpeername, i.e., the destination is the same as the peer.

IPTABLE Config:

-A PREROUTING -p tcp -m socket -j DIVERT 
-A PREROUTING -s 10.2.0.203/32 -p tcp -m tcp --dport 80 -j TPROXY --on-port 8080 --on-ip 10.2.0.204 --tproxy-mark 0x1/0xffffffff 
-A DIVERT -j MARK --set-xmark 0x1/0xffffffff 
-A DIVERT -j ACCEPT

We have logged the activity and it looks OK. For example, we get the following output:

Socket::acceptConnection: getpeername :  10.2.0.203:48517
Socket::acceptConnection: getsockopt :  10.2.0.203:48517

Yet the log from IPTables shows the IP addresses as being correct:

Jul  9 17:37:06 2U-204 kernel: [258876.105481] IN=eth3 OUT= MAC=00:1b:21:61:03:99:00:1b:21:61:c0:70:08:00 **SRC=10.2.0.203 DST=192.168.200.206** LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=56054 DF PROTO=TCP **SPT=48517 DPT=80** WINDOW=17896 RES=0x00 SYN URGP=0
Jul  9 17:37:06 2U-204 kernel: [258876.105697] IN=eth3 OUT= MAC=00:1b:21:61:03:99:00:1b:21:61:c0:70:08:00 **SRC=10.2.0.203 DST=192.168.200.206** LEN=52 TOS=0x00 PREC=0x00 TTL=64 ID=56055 DF PROTO=TCP **SPT=48517 DPT=80** WINDOW=35 RES=0x00 ACK URGP=0

I'm really stumped on this.

score 2 · Answer 1 · answered Oct 06 '12 at 16:21

2

Why are you using getsockopt( ... SOL_IP, SO_ORIGINAL_DST ... )? AFAIK SO_ORIGINAL_DST is intended to be used with the NAT REDIRECT target not TPROXY.

Try a simple getsockname() instead.

answered Oct 06 '12 at 16:21

dongle26

826
1
10
18

getsockopt(...,SO_ORIGINAL_DST,...) occasionally returns client address

1 Answers1