4

I'm building a web service that executes a database process (SQL code to run several queries , then move data between two really large tables), I'm assuming some processes might take 2 to 10 hours to execute.

What are the best practices for executing a long running database process from within a Java web service (it's actually REST-based using JAX-RS and Spring)? The process would be executed upon 1 web service call. It is expected that this execution would be done once a week.

Thanks in advance!

wsb3383
  • 3,841
  • 12
  • 44
  • 59

2 Answers2

6

It's gotta be asynchronous.

Since your web service call is an RPC, best to have the implementation validate the request, put it on a queue for processing, and immediately send back a response that has a token or URL to check on progress.

Set up a JMS queue and register a listener that takes the message off the queue and persists it.

If this is really taking 2-10 hours, I'd recommend looking at your schema and queries to see if you can speed it up. There's an index missing somewhere, I'd bet.

duffymo
  • 305,152
  • 44
  • 369
  • 561
  • thanks! i'm learning ActiveMQ right now to implement this. One more question, is this a situation where I would use pub/sub (topics) or a queue? what are the pros and cons of each in this context? – wsb3383 Aug 12 '10 at 03:12
  • @duffymo: Any idea how to use a token or URL to check progress? – Thudani Hettimulla Mar 21 '14 at 03:49
  • You can either poll ("Done? How 'bout now?") or use eventing and callbacks. – duffymo Mar 21 '14 at 09:10
1

Where I work, I am currently evaluating different strategies for this exact situation, only times are different.

With the times you state, you may be better served by using Publish/Subscribe message queuing (ActiveMQ).

Jordan S. Jones
  • 13,703
  • 5
  • 44
  • 49
  • thank you, why do you think pub/sub is the preferred approach over queuing? – wsb3383 Aug 13 '10 at 00:38
  • The problem really falls into "How do we make this web service asynchronous on the server." There are some solutions for it (Metro WS stack has some support) but, of the ones I've seen, they really aren't all that friendly in their approach. Pub/Sub message queuing was built for situations like this and as the saying goes, "Use the right tool for the job." Sadly, for my work project, we have chosen not to use Pub/Sub message queuing. – Jordan S. Jones Aug 13 '10 at 17:59