How can we implement client-side load balancing with reactive netty?

Question

My microservice is required to direct requests between 2 different servers via TCP connection. Using the current TCPClient, we are required to provide the host & port numbers which means that I can connect to one server TcpClient.create().host(host).port(port)

I did see that netflix ribbon can support tcp but most examples are related to http & eureka

I tried using netflix ribbon where i define the ip address and port number under listOfServers in configuration, and annotate tcpClient bean as load balanced. This does not work. Is there any other solution for client side load balancing or should we create multiple tcp clients and rotate the requests between the different tcp clients?

@Bean
@LoadBalanced
public TcpClient customTcpClient(){
  return TcpClient.create();
}

SERVICE:
  ribbon:
    listOfServers: 123456:1500, 223456:1500

Error: Connection refused. No further information

Note: in the original implementation, it is created with host & port, and then autowired into another class to send request and receive response

EDIT: Based on this discussion, is it right to say that only RestTemplate has support for @LoadBalanced and as such my implementation will not work? https://stackoverflow.com/questions/39587317/difference-between-ribbonclient-and-loadbalanced#:~:text=TL%3BDR%3A%20%40LoadBalanced%20is,is%20used%20for%20configuration%20purposes.

'Connection refused' means that nothing was listening at the IP:port you tried to connect to. — user207421, Aug 10 '23 at 03:59
If I provide the values using the original TcpClient.create().host(host).port(port), the connection is established which means that the IP & port info is correct and the issue is likely in @LoadBalanced does not work together with TcpClient. Examples that I have seen so far use it with a bean returning RestTemplate — roroWorld, Aug 10 '23 at 04:10

score 1 · Accepted Answer · answered Aug 29 '23 at 04:33

Q.) Is it right to say that only RestTemplate has support for @LoadBalanced and as such my implementation will not work?

Yes. @LoadBalanced annotation is primarily designed to work with RestTemplate for HTTP-based communication and doesn't directly support non-HTTP protocols like TCP. In your case you are using a TcpClient for TCP communication, you won't be able to use the @LoadBalanced annotation because it's not designed to work with TCP.

In this scenario, you'll need to manage the load balancing yourself. One approach could be to manually create multiple instances of your TcpClient with different host and port combinations, and then manage the distribution of requests among these instances in your code. This approach would involve rotating the requests between the different instances of your TcpClient to achieve the load balancing.

Approach you can follow:

Create multiple instances of TcpClient, each with a different host and port.
Maintain a collection (e.g., a list) of these TcpClient instances.
In your application code, when you need to make a TCP connection, rotate through the collection of TcpClient instances to distribute the load.

@Configuration public class TcpClientConfig {

@Bean
public List<TcpClient> tcpClients() {
    List<TcpClient> clients = new ArrayList<>();
    
    // Create and add your TcpClient instances with different hosts and ports
    clients.add(TcpClient.create().host("host1").port(1500));
    clients.add(TcpClient.create().host("host2").port(1500));
    
    return clients;
}

}

@Service
public class TcpLoadBalancer {

    @Autowired
    private List<TcpClient> tcpClients;

    private AtomicInteger counter = new AtomicInteger(0);

    public TcpClient getNextTcpClient() {
        int index = Math.abs(counter.getAndIncrement() % tcpClients.size());
        return tcpClients.get(index);
    }
    
    // Use getNextTcpClient() to select a TcpClient instance for each request
}

Note: I use AtomicInteger in my TcpLoadBalance. The primary use of AtomicInteger is when you are in a multithreaded context and you need to perform thread safe operations on an integer without using synchronized. More info checkout this https://stackoverflow.com/a/4818916/12866947

Thank you for your response! I have implemented the same to distribute the load. Do you have any suggestions on how we could integrate this with a liveness check? e.g. if the connection is failing for one of it, all of it should go to the other? — roroWorld, Sep 02 '23 at 06:32

How can we implement client-side load balancing with reactive netty?

1 Answers1