User state in a big (high traffic) application

Question

Assumptions -

1. There are 4 servers sitting behind a reverse proxy which acts as a load balancer
2. Load Balancer is purely load balancing and sends a request to any of the 4 servers depending on their current load
3. Users need to be authenticated for accessing this application, and some space should hold state of all users, as reverse-proxy is only load balancing
4. Application needs to scale beyond 4 servers, say to 4000 servers.

Question -

1. In a large scale multi-server system who holds state of all the users - load balancer, each server, separate server?
   
2. Is the state of all users saved on all servers so that the load balancer can send a request to any server? How does this scale to 100m users?

Answer 1

You can use sticky sessions. It enables the load balancer to bind a user's session to a specific instance. This ensures that all requests from the user during the session are sent to the same instance. Read Sticky and NON-Sticky sessions.

Also let's say the instance gets killed due to some reason, in order to maintain the state, the authentication token and other information can also be saved on a separate redis cache, which is much faster to query. Read Session Management in microservices

Answer 2

1. In a stateless multi-server system, a separate server (authentication server) or a separate server cluster (authentication api) holds state of all users. If its a single authentication server for a large application, you can expect it to have RAM in the range of 100's of GBs, maybe more.
   
   
2. Nope, state of all users is usually not replicated on all application servers, it will be a huge waste of resources. Authentication server (or server cluster) may act as load balancer itself or forward all requests to a separate load balancer - true for a stateless application.
   
   

In a stateful application, individual servers hold state of users through sticky sessions.

If possible, try to keep your application stateless. A stateless application will have better performance and will be easier to scale out than a stateful application!