2

I followed the instructions from this link: How do you get Amazon's ELB with HTTPS/SSL to work with Web Sockets? to set up ELB to work with Websocket (having ELB forward 443 to 8443 on TCP mode). Now I am seeing this issue for wss: server sends message1, client does not receive it; after few seconds, server sends message2, client receives both messages (both messages are around 30 bytes). I can reproduce the issue fairly easily. If I set up port forwarding with iptable on the server and have client connecting directly to the server (port 443), I don't have the problem Also, the issue seems to happen only to wss. ws works fine.

The server is running jetty8.

I checked EC2 forums and did not really find anything. I am wondering if anyone has seen the same issue.

Thanks

Community
  • 1
  • 1
user697620
  • 41
  • 4

2 Answers2

2

From what you describe, this pretty likely is a buffering issue with ELB. Quick research suggests that this actually is the issue.

From the ELB docs:

When you use TCP for both front-end and back-end connections, your load balancer will forward the request to the back-end instances without modification to the headers. This configuration will also not insert cookies for session stickiness or the X-Forwarded-* headers.

When you use HTTP (layer 7) for both front-end and back-end connections, your load balancer parses the headers in the request and terminates the connection before re-sending the request to the registered instance(s). This is the default configuration provided by Elastic Load Balancing.

From the AWS forums:

I believe this is HTTP/HTTPS specific but not configurable but can't say I'm sure. You may want to try to use the ELB in just plain TCP mode on port 80 which I believe will just pass the traffic to the client and vice versa without buffering.

Can you try to make more measurements and see how this delay depends on the message size?

Now, I am not entirely sure what you already did and what failed and what did not fail. From the docs and the forum post, however, the solution seems to be using the TCP/SSL (Layer 4) ELB type for both, front-end and back-end.

Dr. Jan-Philip Gehrcke
  • 33,287
  • 14
  • 85
  • 130
  • So now does the client connect directly to an EC2 instance's IP or to the ELB's IP? If to the ELB's IP, doesn't that make the ELB a bottleneck? – xrDDDD Aug 17 '13 at 00:21
  • A load balancer typically does nothing CPU-intense. The whole point of a load balancer is to just steer network traffic into the right direction. The only bottleneck it theoretically has is the network bandwidth. In case of ELB, I suppose Amazon makes sure that this will never be your bottleneck. If not, there probably is a monitoring feature for your ELB where you can see whether you approach a limit or not. I guess it's not so easy to overload an ELB, they probably have huge ports. – Dr. Jan-Philip Gehrcke Aug 17 '13 at 17:05
  • Do you suppose that multiple ELBs in front of one instance could lead to a bottleneck? First of all, a stacking many ELBs would be just wrong. However, no, even then probably they are not the bottleneck. What do you have to support your assumption? – Dr. Jan-Philip Gehrcke Aug 20 '13 at 08:31
0

This resonates with "Nagle's algorithm" ... the TCP stack could be configured to bundling requests before sending them over the wire to reduce traffic. This would explain the symptoms, but worth a try

KRK Owner
  • 762
  • 7
  • 16