Non-blocking I/O issue

Question

I don't have much experience with asynchronous I/O on sockets. Recently I had to implement a TCP client that would connect to a server in a seperate thread and wait for input data and that in a non-blocking manner so that the thread could be terminated immediately by simply setting some flag varaible from the control thread. My solution was setting socket non-blocking and implementing a loop that called recv over and over again if errno was E_WOULDBLOCK and exited loop when flag variable run wasn't set anymore. It was like that:

while(run) {
   if( -1 == recv(...))
   {
       if(errno == E_WOULDBLOCK)
            continue; 
       else
            break; 
   }
   else
   {
       process_data(...); 
   }
}

The question is: how good is this solution in terms of CPU usage? Will using select/poll/epoll be more effective?

And another one question: is there any way to make blocked select, read or connnect call return immediately from other thread?

It shall better be: `if ((errno == E_WOULDBLOCK) || (errno == EAGAIN))` — alk, Jan 10 '15 at 15:10
@alk, [sometimes](http://stackoverflow.com/q/7003234/132382), yes. — pilcrow, Jan 10 '15 at 15:55
Beware, that on Linux, [recv may block after select](http://stackoverflow.com/questions/5351994/will-read-ever-block-after-select), therefore use both non-blocking IO and select. — Basilevs, Jan 10 '15 at 16:04

alk · Answer 1 · 2015-01-10T15:14:24.630

2

how good is this solution in terms of CPU usage?

Busy waiting is very inefficient.

Will using select/poll/epoll be more effective?

Yes.

is there any way to make blocked select, read or connnect return immediately

Yes, send the blocked thread a signal. The function in question will return -1 and errno shall be set to EINTR. When setting up the signal handler take care of how SA_RESTART is handled by your compiler's implementation.

edited Jan 10 '15 at 15:14

answered Jan 10 '15 at 15:00

alk

69,737
10
105
255

Using a signal to wake up another thread may be unsafe if you don't know the state that the other thread is in. If the other thread happens to /not/ be blocked on the `select`, the signal may break whatever is actually happening. – user3553031 Jan 10 '15 at 15:05
On might very well setup the signal handler before entering the section expected to block and disable the handler on leaving the this section. @user3553031 – alk Jan 10 '15 at 15:13

edmz · Accepted Answer · 2015-01-10T15:19:58.640

The question is: how good is this solution in terms of CPU usage? Will using select/poll/epoll be more effective?

It is terribly inefficient. It is probably the worst way you handle multiple connections. Consider that each recv is a system call: it requires a context switch. A context switch is not expensive, but computing a context switch in a high frequency loop will likely full the CPU usage. Also, if you were to sleep between a call and another, to "soften the loop", you would end up paying a delay between data receive and processing; the more connections you have, the higher it'll be felt.

select basically tells the kernel: "I want these sets of fds to be monitored by you and also be signaled as soon as something happens.". Your process will enter a waiting state and become ready when an event is delivered. Meanwhile, the CPU may serve other processes.

You may use a blocking multiplexer and handle each work in a separate thread (maybe from a thread pool), which is very efficient. But it depends on what you're doing.

And another one question: is there any way to make blocked select, read or connnect call return immediately from other thread?

Well, yes. But is something you want to have? As @alk said, you can send them a signal. In addition, for recv and connect, you might want to use select/poll or any other multiplexer with timeout properties; it might be harder to program, though.

score 0 · Answer 3 · answered Jan 10 '15 at 15:02

The code that you give uses a tight poll loop (ie, it doesn't block until input is received or even wait between polls). Since the CPU is constantly checking for input, it is likely to be very inefficient. Generally, polling like that is only appropriate if the maximum time until input is available is tightly bounded; in the vast majority of cases, you should block using something like select or epoll.

If you have one thread that's blocked on a select (or similar), one easy way to force it to resume from a different thread is to create a pipe, add the read end of the pipe to the select, and write to the pipe from another thread whenever you want to unblock the select thread.

score 0 · Answer 4 · answered Jan 10 '15 at 15:02

Assuming you are programming in C and have a decent kernel version, pselect is the way to go.

Waiting for input and signals simultaneously is a well-known problem and there are many good discussions about the problems and solutions associated with it. You might find this discussion a good start.

Non-blocking I/O issue

4 Answers4