How does a thread pool allows me to handle many client connections?

Question

I want to handle 300 to 400 client connections, but I do not want to create a thread for each client connection (or is there anything wrong in creating a 400 threads?).

so I have read that I should use a thread pool to fix this problem, but I am unable to understand how does a thread pool actually fix this problem. I mean in my understanding of a thread pool, there is a limited number of threads that start to take tasks. But once a thread takes a recv() task it will immediately block if there is nothing to read! so isn't the solution should be that I should have a mechanism that allows me to know if there is something to be read before actually attempting to read it? So how exactly does a thread pool fix my problem of handling many client connections?

Edit: Changed read() to recv().

See this answer. I think it solves your reading problem. http://stackoverflow.com/questions/5616092/non-blocking-call-for-reading-descriptor — , Feb 20 '15 at 08:27

score 2 · Answer 1 · answered Feb 20 '15 at 08:13

2

Creating 300 - 400 threads should work, but isn't the best solution. Context switch is here the keyword you have to search for. Context switches are expensive. Another problem with more threads is that each thread gets 1 MB stack memory, and that memory is limited. You could easily try this an check how many threads you can create.

With a threadpool you have one thread which receives a request an then give these request to your threadpool to work. So you wouldn't have a thread which blocks while waiting for reads. You threadpool is just working when there is something to read.

Another, better option would be on Windows I/O completion ports. Similar technics are also available on linux.

answered Feb 20 '15 at 08:13

user743414

936
10
23

So there is a mechanism that allows the thread pool to know that there is something to be read before attempting to read it? – user4582812 Feb 20 '15 at 08:19
LIke the other user's wrote. You can use select() in your single thread and then "post" something to your thread to work on. – user743414 Feb 20 '15 at 08:23
1

The preferred way on Windows is the other way around, though: Kick off a number of overlapped reads and block on a completion port to get notified whenever a read is done. That works well with one thread or with a pool of threads, and scales to tens of thousands sockets. Under Unix-like systems, you would first `poll` (or `select`) whether there's a socket ready, and then read from it. Works under Windows, but not so well. – Damon Feb 20 '15 at 10:03

Martin Schlott · Accepted Answer · 2015-02-20T09:15:53.250

As user743414 already pointed out, to many threads are not a good idea. But the main problem lies IMHO in your blocking read. You only should use read if there is something to read. Use select to find out which socket has something to read and dispatch that to a worker thread out of the threadpool is the usual way. With Windows you should use WSASockets.

You use selectin a single thread. Than you use the result of select (which will tell you on which socket are action needed) to dispatch the connection to a worker thread.

You wrote that you use microsoft. Take the sample:

https://msdn.microsoft.com/en-us/library/windows/desktop/ms742219(v=vs.85).aspx

search for the code

    //-----------------------------------------
    // If data has been received, echo the received data
    // from DataBuf back to the client
    iResult =
        WSASend(AcceptSocket, &DataBuf, 1, &RecvBytes, Flags, &AcceptOverlapped, NULL);
    if (iResult != 0) {
        wprintf(L"WSASend failed with error = %d\n", WSAGetLastError());
    }

you would replace this part with your threadpool like (pseudocode):

mythreadpool *thread=takeOrCreateThreadFromThreadPool();
thread->callWith(&DataBuf,&RecvBytes);

You will find many different but good threadpool implementation which will use methods like that.

So a thread pool is usually used with `select()`? I thought that `select()` and thread pools are two separate ways to go! I have also read that `select()` can only handle a limited number of sockets. — user4582812, Feb 20 '15 at 09:01
Sorry I wrote "read()" in my question, I meant "recv()" (but I think they mean the same thing). If I don't care much about performance, can I just use `select()` without the thread pool (I don't have constant data arriving from clients, only on long intervals I have data arriving)? — user4582812, Feb 20 '15 at 09:27
@user4582812 read is okay :-). Yes of course. Threadpools add much complexity, you should avoid them. If you do not have complex calculation or DB request which blocks your programm, you should not use multithreading. — Martin Schlott, Feb 20 '15 at 09:31

score 1 · Answer 3 · answered Feb 20 '15 at 08:22

1

The thread pool helps because you probably will not have all the 400 connections constantly sending and receiving data, so your app needs only a handful of threads to manage them all.

A single thread can monitor all the connections (using select for instance) and as soon as select unlocks then it loops through all the sockets that need attention and pass them to the thread pool. If select specified that a sockets has received data then read will not block (and you can still set the timeout of read to 0)

answered Feb 20 '15 at 08:22

Paolo Brandoli

4,681
26
38

So a thread pool is usually used with `select()`? I thought that `select()` and thread pools are two separate ways to go! I have also read that `select()` can only handle a limited number of sockets. – user4582812 Feb 20 '15 at 08:38
@user4582812 select can run in one thread because all it does is wait for data to arrive: when the data arrives then the read() is executed in another thread because it may involved some heavy lifting (like parsing data, storing it somewhere, etc). The maximum number of files descriptor for FD_SET is configurable before importing winsock2.h – Paolo Brandoli Feb 20 '15 at 09:16

How does a thread pool allows me to handle many client connections?

3 Answers3