7

I've got an error 502 when I use google cloud balancer with CDN, the thing is, I am pretty sure I must have done something wrong setting up the load balancer because when I remove the load balancer, my website runs just fine.

This is how I configure my load balancer here

When checking load balancer log message, it shows an error saying failed_to_pick_backend . I have tried to re-configure load balancer but it didn't help.

I just started to learn Google Cloud and my knowledge is really limited, it would be greatly appreciated if someone could show me step by step how to solve this issue. Thank you!

Huy Nguyen
  • 71
  • 1
  • 4
  • Are your backends configured for HTTPS and have SSL certificates installed? In most situations, you only need to configure HTTP (port 80). The frontends will handle HTTPS for you if configured. In your case, your backends are not responding correctly on port 443 (HTTPS). – John Hanley Aug 19 '20 at 22:05

3 Answers3

4

Posting an answer - based on OP's finding to improve user experience.

Solution to the error 502 - failed_to_pick_backend was changing Load Balancer from HTTP to TCP protocol and at the same type changing health check from HTTP to TCP also.

After that LB passes through all incoming connections as it should and the error dissapeared.

Here's some more info about various types of health checks and how to chose correct one.

Wojtek_B
  • 4,245
  • 1
  • 7
  • 21
2

The error message that you're facing it's "failed_to_pick_backend". This error message means that HTTP responses code are generated when a GFE was not able to establish a connection to a backend instance or was not able to identify a viable backend instance to connect to

I noticed in the image that your health-check failed causing the aforementioned error messages, this Health Check failing behavior could be due to:

  • Web server software not running on backend instance

  • Web server software misconfigured on backend instance

  • Server resources exhausted and not accepting connections:

      - CPU usage too high to respond
  - Memory usage too high, process killed or can't malloc()
  - Maximum amount of workers spawned and all are busy (think mpm_prefork in Apache)
  - Maximum established TCP connections

Check if the running services were responding with a 200 (OK) to the Health Check probes and Verify your Backend Service timeout. The Backend Service timeout works together with the configured Health Check values to define the amount of time an instance has to respond before being considered unhealthy.

Additionally, You can see this troubleshooting guide to face some error messages (Including this).

roberto_h
  • 79
  • 2
  • Thank you @roberto_h, I did change the timeout but it didn't help, the running services were not responding with a 200 to the Heath Check probes. It turned out that I set up the load balancer's back end service incorrectly, I change the LB protocol from HTTP to TCP and also change the health check to TCP health check and it is working fine now. Anyway, thank you for your help. Best ! – Huy Nguyen Aug 31 '20 at 22:50
  • I'm getting the same error, but to my understanding internet network endpoint group should not need a health check altogether. – Ville May 06 '22 at 17:23
1

Those experienced with Kubernetes from other platforms may be confused as to why their Ingresses are calling their backends "UNHEALTHY". Health checks are not the same thing as Readiness Probes and Liveness Probes.

Health checks are an independent utility used by GCP's Load Balancers and perform the exact same function, but are defined elsewhere. Failures here will lead to 502 errors. https://console.cloud.google.com/compute/healthChecks

I'll Eat My Hat
  • 1,150
  • 11
  • 20