0

I am trying to execute HTTP requests on a target in parallel using Apache HTTP Client and Spring's async feature.

Here is the Async configuration:

@Configuration
@EnableAsync
public class BackOfficeConfig {
    @Bean(name = "junctionNodeTaskExecutor")
    public Executor junctionNodeTaskExecutor() {
        int nThreads = Runtime.getRuntime().availableProcessors() * 100;
        return new ConcurrentTaskExecutor(Executors.newFixedThreadPool(nThreads));
    }
}

Now in a service, I have the following methods:

@Async("junctionNodeTaskExecutor")  // Async methods have to be public
public void checkJunctionNode(JunctionNode junctionNode) {
    JunctionNodeChecker junctionNodeChecker = new JunctionNodeChecker(junctionNode);
    Instant now = Instant.now();

    // Details dealing with sending the HTTP request
}

private void collectJunctionNodes() {
    logger.info("Inside add Junction Node");
    List<JunctionNode> junctionNodeList = new ArrayList<>();
    int count = 0;
    while (count < 100) {
        JunctionNode junctionNode = (JunctionNode) rabbitMQService.getFromQueue(RabbitMQConfig.QUEUE_CHECK_JUNCTION_NODE);
        if (junctionNode == null)
            break;
        junctionNodeList.add(junctionNode);
        count++;
    }
    logger.info("Collected JunctionNode count: {} for checking", count);
    junctionNodeList.forEach(this::checkJunctionNode);
}

The method collectionJunctionNodes, collect the node objects from a rabbitMQ queue 100 at a time. In the logs I can see the following message:

Collected JunctionNode count: 100 for checking

This is as expected. Now I have 100 nodes but when I send those 100 to be executed in parallel using the forEach stream statement in the line below, I see in the logs of checkJunctionNode that each node is being checked in 5-6 seconds. No parallel execution.

What is going wrong?

conquester
  • 1,082
  • 2
  • 21
  • 44
  • `@Async` works only (as expected), when you access it "as public facade", it is not considered for "private method invocations" (`this.checkJunctionNode()` won't work async-ly)... solution: Move `collectJunctionNodes()` to another class/bean, then `(public) checkJunctionNode` will be executed asynchronously.. – xerx593 Feb 22 '19 at 16:04
  • see: https://stackoverflow.com/a/22561903/592355 ... (possible duplicates: https://www.google.com/search?q=spring+%40async+not+working) – xerx593 Feb 22 '19 at 16:09
  • Possible duplicate of [Spring Boot @Async method in controller is executing synchronously](https://stackoverflow.com/questions/29284008/spring-boot-async-method-in-controller-is-executing-synchronously) – xerx593 Feb 22 '19 at 16:13
  • @xerx593 `checkJunctionNode` is public as code posted by OP. Also if you use the method like `junctionNodeList.forEach(this::checkJunctionNode);` it never actually calls `this.checkJunctionNode` – Amit Bera Feb 22 '19 at 16:18
  • ..."being public" is not enough, calling it through (public non static) bean facade is the point, @AmitBera ..and `this:checkJunctionNode`, doesn't meet this requirement. – xerx593 Feb 22 '19 at 16:21
  • ..it behaves similar to `@Transactional` annotation: No effect, when `this.someTxMethod()` ... but `someBean.someTxMethod()`. – xerx593 Feb 22 '19 at 16:23
  • 1
    ..and i would insist, that `this.checkJunctionNode()` is called (as often as) `junctionNodeList.size()` times ;) – xerx593 Feb 22 '19 at 16:26
  • @xerx593 Can you put some more light on what you mean by calling it junctionNodeList.size() times? – conquester Feb 22 '19 at 17:11
  • `junctionNodeList.forEach` does ... "things" ... `n` times.... and `n == junctionNodeList.size()`... – xerx593 Feb 22 '19 at 17:14
  • 1
    ...but this is really beside the issue! ...your (posted) issue can be solved with/duplicates https://stackoverflow.com/a/22561903/592355 – xerx593 Feb 22 '19 at 17:16

0 Answers0