Kubernetes ingress controller proxy requests between regions

Question

Does traefik / traefik mesh can handle multi regional arquitecture to solve request to the right user region?
Any other self hosted solution recomendation?

Rules:

Each region is independent and one region downtime does'nt affect other region
User authentication metadata is replicated across all regions
Requests with an authentication token can enter the network from any region and be verified
The default entry point for authentication request is usa-region-1
The ingress controller / service mesh determines who the user is and in which region their account is hosted
The authentication service add "user_region" headers to the request and "precedence_region" to prevent internal loop
Detects that destionation region is not current region and proxies the request to the correct region with extra headers added

Diagram:

                                   ┌───────────────────────────────────────────────┐
                                   │                                               │
                                   │ usa-region-1                                  │
                                   │                                  ┌───►/app-1/*│
                                   │                                  │            │
                               ┌───┼──► Load  ──────► ┌── Ingress ────┤            │
                               │   │   Balancer       │ Controller    ├───►/app-2/*│
                               │   │                  │     │         │            │
                               │   │                  │     │         │            │
                               │   │                  │     │         └───►/app-3/*│
                               │   │                  │     │                      │
                               │   │                  │     │                      │
                               │   │                  │     └─Authentication       │
 User──────► Cloudflare ──────►│   │                  │          Service           │
Request                        │   │                  │                            │
                               │   └──────────────────┼────────────────────────────┘
                               │                      │ *proxie the request*
                               │   ┌──────────────────┼────────────────────────────┐
                               │   │                  │                            │
                               │   │ europe-region-1  │                            │
                               │   │                  │               ┌───►/app-1/*│
                               │   │                  │               │            │
                               └───┼──► Load ───────► └── Ingress ────┤            │
                                   │   Balancer         Controller    ├───►/app-2/*│
                                   │                        │         │            │
                                   │                        │         │            │
                                   │                        │         └───►/app-3/*│
                                   │                        │                      │
                                   │                        │                      │
                                   │                        └─Authentication       │
                                   │                             Service           │
                                   │                                               │
                                   └───────────────────────────────────────────────┘

Are you aware of latency and cost differences between inter-zone, cross-zone and cross-regions traffic? — Thomas, Jul 09 '22 at 21:25
Yes, traffict between inter-regions only will occour on requests where the user is not identified yet (auth service), after that every request from the user will arrive to their specific region. — joepa37, Jul 11 '22 at 03:48
How will you make sure the user requests the correct region in face of a downtime? — Thomas, Jul 11 '22 at 08:58
For now, downtime affects all users in that region. In the future the plan is to have replication regions at the same location area (usa-region-2, europe-region-2) — joepa37, Jul 11 '22 at 15:53

Gary Archer · Accepted Answer · 2022-07-17T14:20:45.270

LOCAL DATA STORAGE REQUIREMENT

In some markets, there could be regulatory reasons for keeping a user's data stored within their home region. So if a European user has travelled to the USA, they should continue to be routed to European servers, and their data will remain in Europe.

SIMPLE ROUTING BASED ON HEADERS

First you need something in each HTTP request, such as the user_region header that you mention. The ingress controller is the logical place to do this, rather than for each service. Here is a Traefik header based routing example.

ROUTING IN INGRESS CONTROLLERS

If you need more complex logic, then a plugin can be written, using either the extensibility features of the ingress controller, or a dedicated API gateway placed in front of APIs. For an example see this NGINX configuration file, which sets the target host name based on this Lua plugin.

FURTHER INFO

These links may also be of interest. In particular note how unauthenticated requests can be handled, by identifying the user first, then transferring the user to their home region so that authentication takes place there. This can allow credentials for users to only be stored in their home region.

Thanks a lot for the links, that’s exactly my escenario – joepa37 Jul 17 '22 at 20:42 — joepa37, Jul 17 '22 at 20:42

score 0 · Answer 2 · answered Jul 11 '22 at 17:47

What you are describing is generally referred to as "Global Server Load Balancing". While Traefik does not support such a feature on it's own it can be leveraged to provide Global Server Load Balancing in combination with an advanced DNS provider.

In practice you would want to set up location based DNS settings to refer a user to the nearest Ingress Controller instead rerouting between you ingress controllers. This article from nginx details how to set up Global Server Load Balancing with NS1 as a DNS provider and can be adapted for Trafik.

Cloudflare is in front of clusters, but I'm looking for a kubernetes ingress/mesh solution with multi-cluster communication. I updated the diagram to be more accurate. — joepa37, Jul 11 '22 at 19:16

Kubernetes ingress controller proxy requests between regions

2 Answers2