Why Use Gevent Pool to Manage Greenlet Connections in a Server?

Question

I am working with a Python server which spawns a greenlet for each connection to the server. Currently, the server doesn't make use of a greenlet pool. While it was my hunch that using a pool would improve performance (mainly response time and requests-per-second throughput), in my trial-and-error implementing a pool of greenlets, there doesn't seem be much performance benefit over just using Gevent.spawn() for each greenlet/connection.

I have seen this question, which is helpful, although I am curious about the application of a greenlet pool, like Gevent Pool, in a server. Is this a useful pattern, a la thread pool? Or, does using a Pool not matter in the case of a server, since Greenlets are so lightweight compared with threads?

Answer 1

Greenlets are lightweight but they do consume memory. So, even though the number of greenlets a process can support is going to be much larger than the number of threads the OS can support, there is still a cost to them. So a pool is still a useful tool for limiting the number of greenlets that can be spawned - but its size would likely be best set considerably larger than a limit for actual threads would be.

Also, due to their cooperative multitasking nature, the latency on each request (assuming each new request is handled by a new greenlet) would start to rise as the number of greenlets increases beyond a certain threshold. There's a tradeoff between allowing more requests at once and creating poor UX when each request takes an increasing amount of time to complete. It's sometimes better to cap your incoming load and reject new requests - and a pool is a useful way to do that.