I am giving an answer to my own question as I have figured out the issue. The thread deadlock was the root cause of the issue. Multiple threads were calling the same methods concurrently and the threads were getting trapped when waiting for each other to release the lock. Therefore the execution was becoming unresponsive. To fix it, I changed all the methods used in call method of Callable to have synchronisation using keyword "synchronized" in method declaration. It is now working fine locally and in the Cloud environment. The only thing, I still don't understand is why deadlock was only happening in cloud environment but not locally. I guess it might be the different JVM configuration in the cloud.
@Artyom Kovalyov, here is how I splited tasks for parallel processing.
final int numberOfWorkers = 4;
List<Callable<Void>> tasks = new ArrayList<Callable<Void>>();
//code here to create and add tasks to list
final List<List<Callable<Void>>> dividedTasks = Lists.partition(tasks, numberOfWorkers);
final ExecutorService executor = Executors.newFixedThreadPool(numberOfWorkers);
for (List<Callable<Void>> subsetOfTasks : dividedTasks) {
try {
// you can invoke all tasks in one go but I prefer invoking number of taks same as thread pool max thread number
executor.invokeAll(subsetOfTasks);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
executor.shutdownNow();
