Decouple web services from other backend heavy computing service in Java

Question

Background of the web application:

I am using java/spring-mvc/tomcat to provide my web service as well as exposing my restful API to mobile clients. I am happy with everything on the web surface right now. The problem is that my application has a really heavy computing process at its core, which invokes a separate Java program to process the images and return computed data back to the web service. It sometime eats up lots of my EC2 instance memory, or causes an exception that shuts down my Tomcat7 server.

Question:

Right now everything is running under same tomcat7 container, and I am seeking a solution to decouple those two so that I can install them in different server, perhaps find a high memory server for computing program alone.

What are the options out there that allow me to decouple them and improve scalability and stability?

Update:

I can invoke computing engine programmatically or from command line.

Update2:

I have done some researches based on the answer. When I read on another post about What exactly is Apache Camel?, I feel I should probably learn a little more about EIP patterns. Hopefully, it is not overkill.

Solution based on suggestion

After reading through the EIP concept, camel in action, activemq, I finally come up with a solution. It might not be elegant, but it's working. Suggestion and comments would be appreciated! I wrote a queue router based on apache-camel , connecting to activemq broker and running as standalone program in one server. The computing engine running in standalone container and the router is responsible to process jms requestor from my spring container in web server. Later on I just need to config load balance for computing engine from camel if further intensive computing is needed.

@ryo can you *cache* the pictures and compute them, when there is no load on the server? — Eugene, Aug 28 '12 at 21:20
@Eugene,thanks for the suggestion. Yes, I can do that. However, I still have the problem with the stability, sometime running the computing program would take down the tomcat. If there is a way to decouple it , I can just throw some exception to main server and main server alter request to other computing server. — ryo, Aug 29 '12 at 00:32

score 1 · Answer 1 · answered Aug 28 '12 at 21:13

1

The one which are pointing right now is adding more hardware. You need to think through if this solves your problem. Eg: If you are using a 32 bit JVM there are limitations on how much heap size you can specify. If you are lucky to have a 64 bit JVM them then you will have a bigger room for memory. But there is always the possibility of using too much CPU where your application becomes unresponsive.

I prefer breaking the compute intensive tasks into jobs and work them out in a seperate JVM. Persist your jobs in a datastore/JMS so that they do not get lost. Be careful if you are doing DB updates from those jobs to avoid any locking.

answered Aug 28 '12 at 21:13

randominstanceOfLivingThing

16,873
13
49
72

Hi Suresh, I am using 64 bit JVM, I have fine tuned JVM for my computing program. Could you elaborate how you breaking tasks into jobs and work them in separate JVM. Thanks – ryo Aug 29 '12 at 00:34
I isolated the batch processable jobs which does not need to be executed in the user thread and ran them on nodes that are not user facing. It means there are atleast two nodes in the cluster. One user facing and the other doing batch processing with the jobs kicked off by a scheduler like quartz. – randominstanceOfLivingThing Sep 03 '12 at 14:53
Another possibility if you kick off the image processing job based some user activated task. Instead of kicking off the image processing job in a java process in the same machine, post a message to a JMS queue and have JMS listeners that process the image information outside your machine. – randominstanceOfLivingThing Sep 03 '12 at 15:00
yes, I think those are the alternative. i will research on both to see the possibility, thank you! – ryo Sep 04 '12 at 03:12

score 1 · Answer 2 · edited Mar 11 '13 at 22:22

1

If I understand correctly, it seems you need a load balancer.

Have a load balancer to route to one of multiple instances of your webservice/compute engine. You can achieve this using an esb, routing engine, clustered, master-slave, distributed-cache etc most of them interrelated. And you can also spin up additional nodes realtime on EC2 based on load.
Else, if the task can be broken, then delegate it to multiple nodes/services. You will need some orchestration mechanism.

There are open source solutions that can address 1 and 2 above.

edited Mar 11 '13 at 22:22

nneonneo

171,345
36
312
383

answered Aug 29 '12 at 01:56

techuser soma

4,766
5
23
43

thanks for your suggestion, javausersoma. I am going to use load balancer in my production environment, but I still want to find a way to decouple webservice/compute engine if possible. The computing engine can be broken to different task or running in other JVM. I am interested "delegate it to multiple nodes/services". Could you tell me a little about it, so that I can start searching on google. :) Thanks! – ryo Aug 29 '12 at 13:37
ryo, if you are looking at real-time scenario you can do this break down in the esb itself. If planning to use camel in esb, it provides an option for combining results from mulitple calls, using aggregation and parallel processing. Probably many esbs have similar mechanism. workflow and activity orchestration are some ideas but they are not very realtime. – techuser soma Aug 31 '12 at 20:55
I will read on a little more on camel since it support spring framework. thank you! – ryo Sep 04 '12 at 03:35

score 0 · Answer 3 · edited Mar 11 '13 at 22:23

0

Does the backend work synchronously? I mean, when the mobile clients requests something do they have to wait for the backend to do a lot of processing?

If yes, you can grow horizontally, putting more worker nodes (backend webapps) and a front Nginx or any balancer. It's the fastest way.

Do you have reutilizable data? if yes, you can use something like memcached.

Hope it helps, if you give us more information I'm pretty sure that we will provide better advice.

edited Mar 11 '13 at 22:23