Do Entity Framework async methods consume ThreadPool threads?

Question

I usually use many EF Core async methods in my web application like this:

await db.Parents.FirstOrDefaultAsync(p => p.Id == id);

As we know, initial number of threads in ThreadPool by default is limited to number of CPU logical cores. Also user requests are handled by threads in ThreadPool.

Should I worry about handling user requests or performance issues due to many async calls in my application?

By default the ThreadPool is limited to max 32767 worker threads and max 1000 completion port threads — Sir Rufo, May 25 '20 at 14:22
@SirRufo That numbers are MaxLimit Rufo, but default numbers of threads that generated on startup of application is what I mentioned in question — Arman Ebrahimpour, May 25 '20 at 14:30
Since that's a DB call it's IO bound, so it's going to free the thread while it does the IO with the DB, then it will schedule the code after it to run. That might pick up on the original thread or be picked up by a thread pool thread, but either way this doesn't spawn some new thread to do the DB call. — juharr, May 25 '20 at 14:34
@ArmanEbrahimpour The default numbers of threads generated at startup are start values but **no limits** — Sir Rufo, May 25 '20 at 14:53
Awaiting the async methods on the `DbContext` do **NOT** cause any additional threads, on the thread pool or otherwise, to be used. — Bradley Uffner, May 25 '20 at 15:01
Using an async overload to query the DB will enable the thread that handles the request to serve another request instead of waiting for the results to get back from the DB. This is usually a good thing but may have some pitfalls as mentioned in @David Browne - Microsoft's answer. Once the awaited task finishes, the remainder of the async method is executed on a thread pool thread in the context of an ASP.NET Core app. This obviously requires a free one. In a GUI app, it's executed on the dispatcher thread. — mm8, May 25 '20 at 15:12
Do you always `await` each one EF Core async method individually, like it is shown in your example, or you also store multiple concurrent tasks in a list and then `await` all of them with `Task.WhenAll`? — Theodor Zoulias, May 25 '20 at 16:12
@TheodorZoulias I don't use `WhenAll` because DbContext isn't ThreadSafe and I guess it will cause concurrency issues. But I don't get its importance — Arman Ebrahimpour, May 25 '20 at 16:43

David Browne - Microsoft · Accepted Answer · 2020-05-25T20:36:00.320

7

Should I worry about handling user requests or performance issues due to many async calls in my application?

EF Core provides an Async query interface for repositories. Whether it's async-all-the-way, or whether certian methods block thread pool threads is dependent on the EF provider. SQLServer's SqlClient has task-based Async methods that don't block threads. Most other providers do too. But for instance for the EF in-memory provider, or perhaps the SQLite provider it may be async-over-sync, either completing synchronously and returning a completed Task, or blocking a thread pool thread.

So EF normally won't block your threads. And when you make an Async call to the database it frees your application's thread to do more work. Like handle additional requests. If you have too many concurrent requests to your database, each request will start to take more time.

When this happens you need to have a mechanism to slow down the rate of new requests to the database, otherwise you you'll get into a bad state. EG where the database server is has 2000 running requests, most of which are on behalf of clients who've given up and timed out. And new requests aren't handled in a timely manner because of all the old requests.

Generally throughput increases as you add concurrency up to a point, but beyond that point overall throughput decreases, sometimes drastically. Something like this:

It’s up to you to limit overall concurrency to prevent severe degradation in throughput. It’s better to fail some requests early (eg with an HTTP 503) than accept them all and not complete any within your SLA.

One of the benefits of using synchronous database access is that it occupies an application thread for the duration of the database interaction, automatically adding backpressure to the request flow. Having a request have to wait for a thread pool thread when all of the thread pool threads are busy is actually a good thing. When you go all async this control goes away and you need to think about replacing it.

ASP.NET Core currently has no built-in throttling. Your web server host may have some, and, for instance, SqlConnection's connection pool limit serves to limit the number of concurrent requests per application instance. But you've got to have something that allows you to handle a surge in request volume in an orderly fashion.

edited May 25 '20 at 20:36

answered May 25 '20 at 14:47

David Browne - Microsoft

80,331
6
39
67

Totally agree with David. Another area is the Tcp stack itself. It almost never breaks and will continue to queue requests and responses. At some point both types hit an exponential time curve known as the breaking point. Everything works but major delays are seen. Moral: just because we can doesn't mean we should. – JWP May 25 '20 at 15:06
So your suggestion is to replace the asynchronous requests to the EF with synchronous requests, in order to achieve throttling by `ThreadPool` starvation? I don't think that this is a good suggestion. There should be better ways for database throttling than clogging the system with blocked threads! – Theodor Zoulias May 25 '20 at 16:51
No. The question was what you should worry about going all async to the database. And the answer is that you should worry about having too many concurrent requests to the database, as neither the threadpool nor ASP.NET Core will limit the concurrency. – David Browne - Microsoft May 25 '20 at 17:36
2

I think that the OP is worried about the possibility that the EF is blocking threads under the hood, contrary on how asynchronous methods [are expected to behave](https://blog.stephencleary.com/2013/11/there-is-no-thread.html). And your answer is: *Having a request have to wait for a thread pool thread when all of the thread pool threads are busy is actually a good thing.* Or rephrased: "don't worry if it blocks threads, because having a starved `ThreadPool` is a good thing since it helps at keeping the DB concurrency under control." – Theodor Zoulias May 25 '20 at 18:54
This is a useful and informative clarification, but I still find the statement that ends with the phrase *"is actually a good thing"* quite disturbing. It essentially negates everything I've learned in the last two years about the advantages of asynchrony. Your statement is turning the advantages to disadvantages! – Theodor Zoulias May 25 '20 at 19:09
It's a clear 100% win for client apps with a single dispatcher threads. But for multithreaded server apps it's a double-edged sword. Application server thread pool threads are not an especially-expensive resource in the context of typical "enterprise application" where many requests query a database. For internet-scale web applications much more so. – David Browne - Microsoft May 25 '20 at 19:13
I am still confused. So your claim that having all `ThreadPool` threads blocked is a good thing, applies only to client apps and application servers, and not to web applications? – Theodor Zoulias May 25 '20 at 19:27
Web applications are application servers (the browser is the client). The ThreadPool manages the concurrency. Managing the concurrency is a necessary function, as briefly making a request wait for a a thread to become available is sometimes better than letting it start to execute immediately. If you give the request a thread and it can't get CPU time, or blocks waiting on IO, or a database request, you're just waiting with extra resources. – David Browne - Microsoft May 25 '20 at 19:32
1

But I doubt here because you assume every request goes for database that not true and by starving ThreadPool we starve them too. There should exist some better options – Arman Ebrahimpour May 25 '20 at 19:58
That's one reason ASP.NET core encourages an async model, and relies on the HTTP server (Kestrel, IIS, etc) to limit concurrency. – David Browne - Microsoft May 25 '20 at 20:33
2

I gather that synchronous use of the `ThreadPool` (i.e. blocking I/O) is not a better thing than asynchronous use (i.e. asynchronous I/O). The fact that the former a̶u̶t̶o̶m̶a̶t̶i̶c̶a̶l̶l̶y̶ implicitly imposes some kind of limit on lower tiers is not a benefit over explicitly imposing sensible limits and maybe scale the following tiers. What difference does it make for the DB between a big synchronous web farm or a small asynchronous web farm, if its processing capacity, memory or bandwidth is exceeded either way? However, there's a big infrastructure difference in the web application tier. – acelent May 26 '20 at 14:58
@ArmanEbrahimpour correct. Although it doesn't take a a big web server to support a few hundred threads. The notion that app server threads are expensive is mostly mistaken. A thread is just 1MB of memory and a few handles. – David Browne - Microsoft May 26 '20 at 15:02
3

@DavidBrowne-Microsoft, it depends on scale. Consider IIS 7.5 up to 10, ASP.NET on .NET Framework 4.5 up to 4.8, and an I/O-bound web app or service. Having converted a few of these to asynchronous I/O, each resulted in using 1/500th to 1/1000th of the threads simultaneously. Before, with a TP of max=2000 it was limited to 2000 simultaneous requests, now it can handle 20000+ simultaneous requests while maxing at ~25 threads. The cloud savings due to memory reduction in shared app pools allowed investing on the DB. Even free and shared Azure hosting became options for testing dev versions. – acelent May 26 '20 at 17:15
2

[This blog post](https://learn.microsoft.com/en-us/archive/blogs/rickandy/should-my-database-calls-be-asynchronous) also (originally) recommended the use of synchronous calls. However, the implicit assumption in this answer and that blog post is that there is a single database backend server. In the cloud world, `async` is a much more natural default. – Stephen Cleary May 28 '20 at 02:05
ASP.NET core promotes the async paradigm. So that's probably the way to go. Having a large number of single-tenant databases is is an good argument to limit the concurrency using the per-database-connection-string connection pools, rather than the per-application thread pools. – David Browne - Microsoft May 28 '20 at 03:08
1

@StephenCleary And also I think both of them didn't consider existence of caching mechanisms in high load applications that defends using `async` even in single database cases. Although I got their concern and that seems almost valid. – Arman Ebrahimpour May 28 '20 at 06:37

Do Entity Framework async methods consume ThreadPool threads?

1 Answers1

Linked

Related