ExecutorService slow and hung sometime

Question

I am calling the multiple rest API and for this i am using the ExecutorService for parallel processing. Consumer hit my app more than 10 times every seconds and i observed ExecutorService very slow to response. I have deployed my application on kubernetes using tomcat web server. below is my code , not sure what is causing this to slow.

   ExecutorService WORKER_THREAD_POOL = Executors.newFixedThreadPool(100);

            Collection tasks = new ArrayList();
  for( String str: datalist){
 tasks.add(new MyThreadPool(str)); // MyThreadPool internally calls the REST API using Google HttpRequest.
}
 List<Future<BasePolicy>> futures = null;
            try {
                long startProcessingTime = System.currentTimeMillis();
                futures = WORKER_THREAD_POOL.invokeAll(tasks);
                WORKER_THREAD_POOL.shutdown();

                if (!WORKER_THREAD_POOL.awaitTermination(60000, TimeUnit.SECONDS)) {
                    WORKER_THREAD_POOL.shutdownNow();
                }

                long totalProcessingTime = System.currentTimeMillis() - startProcessingTime;
                log.info("total time to complete the thread pool- " + totalProcessingTime);
            } catch (InterruptedException e) {
                log.error("error occured during ASYNC process ");
                e.printStackTrace();
            }
            log.info("Finished Waiting All threads completed ");
            for (Future<Data> mFuture : futures) {
               // my logic 
            }

I believe may be external service is not responding fastly. Did you check how long does it take to get the response from the external service?. — b.s, Aug 01 '20 at 06:43
@harry , my backend rest services are responding fast I have timer set and verified. If I get one request at a time it works bu only if multiple request comes to my service then I see slow and hung issues — user1591156, Aug 01 '20 at 06:49

score 2 · Answer 1 · answered Aug 01 '20 at 07:22

and i observed ExecutorService very slow to response.

ExecutorService WORKER_THREAD_POOL = Executors.newFixedThreadPool(100);

Let's start with the question: How many cores does your machine have?
For, newFixedThreadPool, it is suggested to start with as many threads as the number of cores in your machine (if the tasks are long running ones). With a value of 100, your CPU is going to be busy with scheduling and context-switching. On top of that, since it's a newFixedThreadPool, so many threads are going to remain in pool even when there is less load - not helpin the CPU.

As suggested in Effective Java, newCachedThreadPool usually does the good thing. Here is more on the Thread Pools.

If you are expecting a really heavy load, I think deployment of apps on multiple servers would be the wise thing to do (this depends on the capacity of your single server). Threads are not going to help and can eventually make your app slow.

Arghya Sadhu · Answer 2 · 2020-08-01T08:29:54.817

When you deploy an application in kubernetes if you do not specify the memory and cpu requirement then kubernetes scheduler schedules the pod into a node on best effort basis which can lead to starvation and even eviction of the pod.

You can help the scheduler to take a better scheduling decision by specifying the memory and cpu requirements as below

apiVersion: v1
kind: Pod
metadata:
  name: frontend
spec:
  containers:
  - name: app
    image: images.my-company.example/app:v4
    env:
    resources:
      requests:
        memory: "64Mi"
        cpu: "250m"
      limits:
        memory: "128Mi"
        cpu: "500m"

This will make sure the pod gets scheduled on a node which can satisfy requirement of starting with 64Mi memory,250m cpu and allow it to burst upto 128Mi memory, 500m cpu.

ExecutorService slow and hung sometime

2 Answers2