Advice around Web Service consuming another Web Service

Question

I'm designing an application and trying to do some research on how it should work and any tips etc that I could use.

I need to develop a middleware Web service running on Tomcat 6.

Client program consumes my webservice.

My Webservice in turns needs to run a number of searches, 10, based on information from the client. These searches are using an 3rd Party web service. 3rd Party supply Java Stub Classes.

Can/Should I write my web service to be multi-threaded so each thread is created and used for a search and results collated and returned to client?

Searches can take a while to complete approx 200-500ms

All advice is gratefully received,

Answer 1

I'm a bit uncertain as to what your needs are exactly. Are the searches able to be run in parallel? If that is the case it may not be a bad idea to use multi-threading for executing them.

We have something similar in an application I'm working on - a long-running search is run in a separate thread, so that other processing can continue, and then when it is finished the results are sent back to the client.

There is no problem with this, we run on Tomcat 6 and it works fine. Obviously the usual caveats with multi-threading apply, we're using the Java 6 java.util.concurrent library which is really useful.

Answer 2

There do seem to be potential benefits here in having several back-end queries running in parallel, so some kind of multi-threading seems like a good idea.

There are a few issues that occur to me:

1. The direct spawning of threads in a Java EE container is not generally recomended - Java EE containers like to be in control of that - so there are specifically supported APIs for doing that in the Java EE world (see this answer for more on this topic.) I don't know whether Tomcat supports such APIs these days, if not then perhaps something like this may work.
2. You need a good strategy to deal will broken and slow responses. Suppose you have 7 out of 8 responses and the 8th seems to take a very long time is it better to give a rapid partial responses. Best to think about this up front.
3. Which leads to is it better to have some kind of "noticebaord" approach, send a request, come back later to collect interim results, come back yet later to collect more complete results.
4. Some back end systems might react badly to excessive requests from the same source. You may need to throttle request freuqencies both to be "sociable" and also to avoid any black-listing policies.