I am trying to execute HTTP requests on a target in parallel using Apache HTTP Client and Spring's async feature.
Here is the Async configuration:
@Configuration
@EnableAsync
public class BackOfficeConfig {
@Bean(name = "junctionNodeTaskExecutor")
public Executor junctionNodeTaskExecutor() {
int nThreads = Runtime.getRuntime().availableProcessors() * 100;
return new ConcurrentTaskExecutor(Executors.newFixedThreadPool(nThreads));
}
}
Now in a service, I have the following methods:
@Async("junctionNodeTaskExecutor") // Async methods have to be public
public void checkJunctionNode(JunctionNode junctionNode) {
JunctionNodeChecker junctionNodeChecker = new JunctionNodeChecker(junctionNode);
Instant now = Instant.now();
// Details dealing with sending the HTTP request
}
private void collectJunctionNodes() {
logger.info("Inside add Junction Node");
List<JunctionNode> junctionNodeList = new ArrayList<>();
int count = 0;
while (count < 100) {
JunctionNode junctionNode = (JunctionNode) rabbitMQService.getFromQueue(RabbitMQConfig.QUEUE_CHECK_JUNCTION_NODE);
if (junctionNode == null)
break;
junctionNodeList.add(junctionNode);
count++;
}
logger.info("Collected JunctionNode count: {} for checking", count);
junctionNodeList.forEach(this::checkJunctionNode);
}
The method collectionJunctionNodes, collect the node objects from a rabbitMQ queue 100 at a time. In the logs I can see the following message:
Collected JunctionNode count: 100 for checking
This is as expected. Now I have 100 nodes but when I send those 100 to be executed in parallel using the forEach stream statement in the line below, I see in the logs of checkJunctionNode that each node is being checked in 5-6 seconds. No parallel execution.
What is going wrong?