2

I have a following nginx setup on my server A (internet facing, only relevant parts):

upstream new_api {
  server unix:///home/ubuntu/new_api/shared/tmp/sockets/puma.sock;
}

server {
  listen 80 default_server;
  listen [::]:80 default_server;

  large_client_header_buffers 4 16k;

  ssl_certificate           /etc/nginx/cert.crt;
  ssl_certificate_key       /etc/nginx/cert.key;

  location ~ (^(/some/location|/some/other)) {
    proxy_pass http://new_api;
  }

  location / {
    proxy_pass https://serverB.com;
  }
}

Now, if I go to /some/location it is served fine with new api upstream. But with anything else I keep getting "400 Bad Request Request Header Or Cookie Too Large" from nginx. Even with curl with no cookies and only two short headers. Increasing large_client_header_buffers does not help.

The interesting part is that I don't see this request coming to Server B at all, so it gets cut off on Server A. Why? Can it be because of https protocol I'm proxy_passing to?

Also, before setting up Server A everything was going to Server B without any problems.

katafrakt
  • 2,438
  • 15
  • 19

1 Answers1

4

It turns out there was some mix-up with domain resolving (which I don't really understand), and as a result request to server B were passed to Server A instead. It kept adding its own IP to X-Forwarded-For header, until it exceeded max size - so the error message was actually correct.

To debug further, I used

tcpdump -n -S -s 0 -A 'tcp dst port 80' | grep -B3 -A10 "GET"
katafrakt
  • 2,438
  • 15
  • 19
  • I am possibly running into a similar issue, can you elaborate on how that tcpdump command helps to debug? I get output from it, but my X-Forwarded-For only show the actual IP I am forwarding for, once. – Evan Morrison Jun 27 '19 at 17:45
  • My problem: X-Forwarded-For: 192.168.100.100, 172.20.0.1, 172.20.0.1, 172.20.0.1, 172.20.0.1, 172.20.0.1, 172.20.0.1, 172.20.0.1, 172.20.0.1 (repeating many, many times). I found out about this running the following command: `sudo tcpflow -p -c -i any port 8000` – Tarida George Jul 29 '21 at 18:25