executing millions of thread concurrently in java

Question

I have a requirement to handle millions of thread and i know its quite dependent on the hardware configuration and jvm. I have used executors for the task

call flow of my project :

user(mobile)----->Server(Telecom) ------>Application----->Server(Telecom)----->User

Code call flow : A------>B---------->C

    //Code snippet of A
    public static final int maxPoolSize=100;
ExecutorService executorCU=Executors.newFixedThreadPool(maxPoolSize);

    Runnable handleCalltask=new B(valans, sessionID, msisdn);
executorCU.execute(handleCalltask);

    //Code snippet of B
    public static final int maxPoolSize=10;
ExecutorService executor=Executors.newFixedThreadPool(maxPoolSize);

    Runnable handleCalltask=new c(valans, sessionID, msisdn);
executor.execute(handleCalltask);

and there are shared map which i implemented as concurrencyHashMap which gets loaded at the loading of application.

Is my approach is correct and if not can anybody suggest how i can achieved maximum threading in my web application.

I have tested with Jmeter and its result are not at all encouraging.

Thanks.

where does the requirement come from? You should probably go to workplace.stackexchange.com and ask "how do I explain to my crazy boss or client that millions of threads is a bad idea" — UmNyobe, Jan 21 '14 at 13:02
Millions of threads -> millions of call stacks -> terabytes of RAM just for them. — Marko Topolnik, Jan 21 '14 at 13:03
Even if you had terabytes of RAM, it would take something much more robust in load balancing than fixed size executor service. — Jakub Zaverka, Jan 21 '14 at 13:07
Further, millions of threads + shared state concurrency... That is certainly a really good way to maximize resource starvation. A `ConcurrentHashMap` will not help you here. If you looking into this seriously then you will **have** to consider something like [Apache Storm](http://storm-project.net/). — Boris the Spider, Jan 21 '14 at 13:15
@BoristheSpider Thanks for the advice but what if i have one hashmap which is loaded at run time and shared among thread.I thought concurrentHashMap will be useful as i have to more reader and comparitively very few writer. — Suresh, Jan 21 '14 at 13:18
Thanks Guys for advice but there are application which support millions of thread then i pondered how they have achieved?just by increasing the hardware configuration. — Suresh, Jan 21 '14 at 13:20
@Suresh Your requirements go beyond anything naive shared state concurrency can do without huge penalties. I suspect the requirement is just misspecified but if not you really need to read up on more advanced concurrency topics. — Boris the Spider, Jan 21 '14 at 13:20
@Suresh - _application which support millions of thread_. Like? — Boris the Spider, Jan 21 '14 at 13:23
Don't confuse millions of concurrent connections/requests with millions of threads. The requests line up in a queue while a modest team of threads serve them. — Marko Topolnik, Jan 21 '14 at 13:25
@BoristheSpider Thanks for the advice but i read about concurrentHashMa in http://stackoverflow.com/questions/510632/whats-the-difference-between-concurrenthashmap-and-collections-synchronizedmap and if not what i should opt if the hashmap is static and loaded at loading time of web application? — Suresh, Jan 21 '14 at 13:32

score 2 · Answer 1 · edited May 23 '17 at 12:21

Is my approach is correct

IMO, no, it's definitely not the correct approach.

and if not can anybody suggest how i can achieved maximum threading in my web application.

Separate receiving messages from the client with processing the messages. That way, you can horizontally scale the two parts independently to meet your requirements without having millions of threads in a single JVM.

A few suggestions:

1) I'd make the web application as light as possible and submit any long running tasks to some sort of backend processor.

Within the same JVM, you could use a ThreadPoolExecutor with an ArrayBlockingQueue.

If you wanted to submit the jobs to another JVM, you could use JMS with competing consumers or something like Apache Kafka.

Again the benefit here is that you can add more nodes to either the backend or frontend of the app as required.

2) If required, make your application server's thread pool larger.

For instance, with Tomcat you'd tweak the parameters described here: http://tomcat.apache.org/tomcat-7.0-doc/config/executor.html. Explaining how to correctly tune these parameters is more than I can describe here. Among other things, the values you select will depend on the average number of concurrent requests, the maximum number of concurrent requests, the time required to serve a single request, and the number of application servers in your pool.

3) You'll get the most scalability by reducing statefulness.

If a request can be dispatched to any front end consumer and then processed by any backend consumer, you can add more instances of either to scale. If one request depends on another, you'll need to synchronize the processing of requests across nodes, which reduces scalability. Design things to be stateless from the start if at all possible.

I have tested with Jmeter and its result are not at all encouraging.

You need to profile your application to determine where the hot spots are. If you follow my recommendations above, you can easily add more horsepower where required.

executing millions of thread concurrently in java

1 Answers1