WCF Multithreading With Scalability Considerations

Question

Working on a stateless WCF rest web service and have an operation with 3 independent tasks. Each one can be run independently. Each task consists of a web service call to an external API & a follow up local DB read operation that takes less than 0.25 sec.

First thing that comes to mind is that I should spawn 3 separate threads then join and return result. Using a Thread Pool would probably not be a good idea here as its limited to 250 treads max.

Performance is of concern, but not at the expense of scalability.

Should I be concerned about the overhead of starting & joining 3 separate threads for each web service call ?

score 0 · Answer 1 · answered Dec 02 '15 at 21:24

0

Wrap the calls to external service into async Task methods, then call from your WCF method. It will use thread pool and will queue your web service calls nicely if thread pull is exhausted.

answered Dec 02 '15 at 21:24

Riad Baghbanli

3,105
1
12
20

But that would go against the scalability requirement. Thread pools are limited to 250 threads max. – AlexVPerl Dec 02 '15 at 21:26
Thread pool size is usually equals number of logical threads multiplied by 25. To scale your service you will need to keep increasing the number of logical threads available to your service. – Riad Baghbanli Dec 02 '15 at 21:36
I dont think ThreadPool is a good fit for the scenario in my question due to # of threads limitation. I would not even consider using ThreadPool for this. Using Threads directly, there is no soft limit imposed, just what your hardware can support. – AlexVPerl Dec 02 '15 at 21:39
Think about it, depending on the CPU (lets say a hex core) ThreadPool limit will be 300-350 threads. 3 threads per service call, thats only ~100 concurrent calls that you can support, rest will be queued. Again, not scalable. – AlexVPerl Dec 02 '15 at 21:42
Threads from ThreadPool which are in await state are returned to ThreadPool until async method comes back. – Riad Baghbanli Dec 02 '15 at 21:50

score 0 · Answer 2 · edited May 23 '17 at 12:15

You can use async IO to perform the webservice calls. Async IO does not occupy any thread at all while it is running. You can do the same thing for the database calls. This alleviates any threading concern that you might have.

Alternatively, you can rely on the thread-pool. You can increase the limits. You can calculate how many threads you need: If 100 requests arrive per second and each one takes 2 seconds to complete you need 200 threads. This can easily be served by the built-in thread pool assuming you configure appropriate limits.

In case the external service is down and takes 30 seconds to timeout this number now shoots up to 3000 threads which I consider unsafe. So you either need a low timeout, a circuit breaker or async IO.

So in order to decide you need to forecast load and latency.

I'll link to some discussion for why and when to use async IO:

https://stackoverflow.com/a/25087273/122718 Why does the EF 6 tutorial use asychronous calls? https://stackoverflow.com/a/12796711/122718 Should we switch to use async I/O by default?

WCF Multithreading With Scalability Considerations

2 Answers2