4

Ignite cluster node does not perform load balancing on multiple instances of a service deployed on it.

I have a single node Ignite cluster, on which I have deployed a service, using deployMultiple method.

final var igniteServices = ignite().services();
igniteServices.deployMultiple("elasticsearchQueryService", new ElasticsearchQueryServiceImpl(), 5, 0);

In the cluster node log, I can see that multiple instances of the service are deployed.

2023-06-06T14:47:39.755+03:00  INFO 19452 --- [ker-#73%ignite%] o.a.i.i.m.d.GridDeploymentLocalStore     : Class locally deployed: class com.azdio.is.ignite.service.search.ElasticsearchQueryServiceImpl
2023-06-06T14:47:39.755+03:00  INFO 19452 --- [ker-#73%ignite%] o.a.i.i.p.s.IgniteServiceProcessor       : Starting service instance [name=elasticsearchQueryService, execId=159d0e8c-88c9-41ae-a8c4-22f8b03d665c]
2023-06-06T14:47:39.756+03:00  INFO 19452 --- [ker-#73%ignite%] o.a.i.i.p.s.IgniteServiceProcessor       : Starting service instance [name=elasticsearchQueryService, execId=d6e5118c-ac78-40d0-a1fc-75a1aa62a235]
2023-06-06T14:47:39.756+03:00  INFO 19452 --- [ker-#73%ignite%] o.a.i.i.p.s.IgniteServiceProcessor       : Starting service instance [name=elasticsearchQueryService, execId=bbf5a1e7-5aec-41f3-a16b-548b6ea741cf]
2023-06-06T14:47:39.756+03:00  INFO 19452 --- [ker-#73%ignite%] o.a.i.i.p.s.IgniteServiceProcessor       : Starting service instance [name=elasticsearchQueryService, execId=fe8b1abe-22fa-429a-a931-20e4105a7178]
2023-06-06T14:47:39.756+03:00  INFO 19452 --- [ker-#73%ignite%] o.a.i.i.p.s.IgniteServiceProcessor       : Starting service instance [name=elasticsearchQueryService, execId=fdc82642-9c1b-492c-8161-7c6beeb67482]

When an instance of my service is created, I assign an instanceId to it, to be able to track which instance gets called. The instanceId is printed in the log:

2023-06-06T14:47:40.012+03:00  INFO 19452 --- [ice-#95%ignite%] .a.i.i.s.s.ElasticsearchQueryServiceBean : [Elasticsearch][QueryService][instanceId: 978ee520-ae4f-4047-a779-7ead8f0233f8][ThreadPoolExecutor initilized]
2023-06-06T14:47:40.022+03:00  INFO 19452 --- [ice-#94%ignite%] .a.i.i.s.s.ElasticsearchQueryServiceBean : [Elasticsearch][QueryService][instanceId: f71b377b-d23a-4523-84a4-80d2410ca085][ThreadPoolExecutor initilized]
2023-06-06T14:47:40.023+03:00  INFO 19452 --- [ice-#96%ignite%] .a.i.i.s.s.ElasticsearchQueryServiceBean : [Elasticsearch][QueryService][instanceId: 0d985409-cfb0-4f56-ab72-6d79a664c5b2][ThreadPoolExecutor initilized]
2023-06-06T14:47:40.023+03:00  INFO 19452 --- [ice-#93%ignite%] .a.i.i.s.s.ElasticsearchQueryServiceBean : [Elasticsearch][QueryService][instanceId: b1bdc35e-3746-46af-99b7-74af6049a0e6][ThreadPoolExecutor initilized]
2023-06-06T14:47:40.023+03:00  INFO 19452 --- [ice-#97%ignite%] .a.i.i.s.s.ElasticsearchQueryServiceBean : [Elasticsearch][QueryService][instanceId: 65a4e01c-1452-4fa3-aab1-3445ce2d1184][ThreadPoolExecutor initilized]

Now, when I obtain a service proxy on my Ignite client node, and I call the service, I observe that there's no load balancing between the multiple instances on the ignite cluster node.

Each call lands on the exact same instance of my service.

The Ignite documentation clearly states: You can have multiple instances of a service on one or multiple nodes.

It is not clear from this documentation if Ignite will load balance between multiple instances of a service on a single node.

When I tested with multiple nodes cluster, the service proxy calls from the client, get load balanced between the multiple notes in the cluster. However when a single node hosts multiple instances of the same service, I cannot observe load balancing.

Is there anything that can be done to achieve load balancing between multiple instances deployed on a single node?

Here's how I obtain the service proxy on my client:

elasticsearchQueryService = cacheFacotry.getIgnite().services().serviceProxy("elasticsearchQueryService", ElasticsearchQueryService.class, false);

Below is the log from multiple executions of a search method in my service. It shows that invocations end up on the same service instance - b1bdc35e-3746-46af-99b7-74af6049a0e6

2023-06-06T14:58:34.824+03:00 DEBUG 19452 --- [vc-#119%ignite%] .a.i.i.s.s.ElasticsearchQueryServiceBean : [instanceId: b1bdc35e-3746-46af-99b7-74af6049a0e6][requestId: 56f8fa50-ed49-451e-af51-d2357574ae13][size: 89/89][time: 7/26]
2023-06-06T14:58:34.840+03:00 DEBUG 19452 --- [vc-#124%ignite%] .a.i.i.s.s.ElasticsearchQueryServiceBean : [instanceId: b1bdc35e-3746-46af-99b7-74af6049a0e6][requestId: 3d4ce908-49fc-431a-89b4-b6e60de3cb47][size: 52/52][time: 6/22]
2023-06-06T14:58:34.841+03:00 DEBUG 19452 --- [vc-#121%ignite%] .a.i.i.s.s.ElasticsearchQueryServiceBean : [instanceId: b1bdc35e-3746-46af-99b7-74af6049a0e6][requestId: 6664cf67-056f-487f-b50a-d260c3a16d08][size: 52/52][time: 9/16]
2023-06-06T14:58:34.841+03:00 DEBUG 19452 --- [vc-#120%ignite%] .a.i.i.s.s.ElasticsearchQueryServiceBean : [instanceId: b1bdc35e-3746-46af-99b7-74af6049a0e6][requestId: 692e5fb3-bdd9-4ea8-be6a-a3f0bdbaf472][size: 89/89][time: 12/23]
2023-06-06T14:58:34.842+03:00 DEBUG 19452 --- [vc-#107%ignite%] .a.i.i.s.s.ElasticsearchQueryServiceBean : [instanceId: b1bdc35e-3746-46af-99b7-74af6049a0e6][requestId: 2bae50d7-ec15-4c4b-a021-099111b3b25c][size: 89/89][time: 13/23]

Any ideas? Thank you.

Assen.

Assen Sharlandjiev
  • 41
  • 2

1 Answers1

0

I guess this is because you set MaxPerNodeCount to 0.

Java doc mentioned:

Deploys multiple instances of the service on the grid. Ignite will deploy a maximum amount of services equal to 'totalCnt' parameter making sure that there are no more than 'maxPerNodeCnt' service instances running on each node.

It can be set via API call or via configuration:

 ServiceConfiguration cfg = new ServiceConfiguration();

 cfg.setName(name);
 cfg.setService(svc);
 cfg.setTotalCount(totalCnt);
 cfg.setMaxPerNodeCount(maxPerNodeCnt);

As I see you set 0. But it means:

Maximum number of deployed service instances on each node, {@code 0} for unlimited.

Try to set maxPerNodeCnt to 1.

Andrei Aleksandrov
  • 686
  • 4
  • 10