Async connect and disconnect with epoll (Linux)

Question

I need async connect and disconnect for tcp client using epoll for Linux. There are ext. functions in Windows, such as ConnectEx, DisconnectEx, AcceptEx, etc... In tcp server standard accept function is working, but in tcp client doesn't working connect and disconnect... All sockets are nonblocking.

How can I do this?

Thanks!

This could help you : http://stackoverflow.com/questions/2875002/non-blocking-tcp-connect-with-epoll — Skippy Fastol, Apr 17 '12 at 08:11
As a possible alternative to the suggestions on the linked DJB page, I'd like to suggest trying to `dup` and `close` the descriptor (and use the duplicate). Not tested, but it should work, in my understanding. The docs state that it is a serious programming error not to check the return value of `close`, because it may return a previous error. That's just what you want (if `close` gives an error, `connect` failed). Though of course if you use `epoll` then you're guaranteed to have an OS where `getsockopt(SO_ERROR)` will just work... — Damon, Apr 17 '12 at 09:08
If viable, the simplest option is to wait until after connect() returns before you set NON_BLOCK. — CodeClown42, Apr 17 '12 at 13:11
@goldilocks: +1 Not asynchronous unless you use a worker thread for that, but I agree the simplicity is tempting. Plus, DNS resolve -- which you likely need -- will need a worker thread anyway unless you want to block on that (`getattrinfo_a` does just that internally, too). So while you block in the worker anyway, you can as well block on connect, too... — Damon, Apr 17 '12 at 14:07
I have 1 work thread for all my needs (tcp server/client, udp socket, timerfd). In this thread I'm using epoll for async work. So I wait for epoll_wait(...) and then do what I need. Forexample: if socket is listening socket - I call accept function, create new client with this socket and add it to epoll queue. But in tcpclient - I can't add it to epoll before connect done... And if I do this - client connects several times (3-4)... — Alexander, Apr 17 '12 at 14:24

score 40 · Answer 1 · edited Jul 15 '14 at 12:44

To do a non-blocking connect(), assuming the socket has already been made non-blocking:

int res = connect(fd, ...);
if (res < 0 && errno != EINPROGRESS) {
    // error, fail somehow, close socket
    return;
}

if (res == 0) {
    // connection has succeeded immediately
} else {
    // connection attempt is in progress
}

For the second case, where connect() failed with EINPROGRESS (and only in this case), you have to wait for the socket to be writable, e.g. for epoll specify that you're waiting for EPOLLOUT on this socket. Once you get notified that it's writable (with epoll, also expect to get an EPOLLERR or EPOLLHUP event), check the result of the connection attempt:

int result;
socklen_t result_len = sizeof(result);
if (getsockopt(fd, SOL_SOCKET, SO_ERROR, &result, &result_len) < 0) {
    // error, fail somehow, close socket
    return;
}

if (result != 0) {
    // connection failed; error code is in 'result'
    return;
}

// socket is ready for read()/write()

In my experience, on Linux, connect() never immediately succeeds and you always have to wait for writability. However, for example, on FreeBSD, I've seen non-blocking connect() to localhost succeeding right away.

@Matt I know it does, you're probably doing something wrong here. What exactly are you trying, where is it failing? Have you put the socket into non-blocking mode using fcntl? — Ambroz Bizjak, Oct 25 '12 at 11:51
Oops, never mind. A bug on my part. Yeah I had non-blocking sockets. I had a bug when checking the result of connect! classic c mistake. — hookenz, Oct 25 '12 at 22:09
Just as an FYI: If non-blocking connect fails, Windows will notify via an exceptional condition rather than socket writability. https://msdn.microsoft.com/en-us/library/windows/desktop/ms740141.aspx — zapstar, Feb 09 '18 at 04:09

score 3 · Answer 2 · answered Aug 15 '13 at 09:14

3

From experience, when detect non-blocking connection , epoll is a little different from select and poll.

with epoll:

After connect() call is made, check return code.

If the connection can not be completed immediately, then register EPOLLOUT event with epoll.

Call epoll_wait().

if the connection failed, your events will be fill with EPOLLERR or EPOLLHUP, otherwise EPOLLOUT will be triggered.

answered Aug 15 '13 at 09:14

Bugs

31
2

Yeah, I forgot to mention in my answer that epoll can return EPOLLERR or EPOLLHUP in addition to EPOLLOUT. Thanks for mentioning, it's corrected. – Ambroz Bizjak Oct 08 '13 at 12:08

Sonny · Answer 3 · 2012-06-06T01:19:32.767

I have a "complete" answer here in case anyone else is looking for this:

#include <sys/epoll.h>
#include <errno.h>
....
....
int retVal = -1;
socklen_t retValLen = sizeof (retVal);

int status = connect(socketFD, ...);
if (status == 0)
 {
   // OK -- socket is ready for IO
 }
else if (errno == EINPROGRESS)
 {
    struct epoll_event newPeerConnectionEvent;
    int epollFD = -1;
    struct epoll_event processableEvents;
    unsigned int numEvents = -1;

    if ((epollFD = epoll_create (1)) == -1)
    {
       printf ("Could not create the epoll FD list. Aborting!");
       exit (2);
    }     

    newPeerConnectionEvent.data.fd = socketFD;
    newPeerConnectionEvent.events = EPOLLOUT | EPOLLIN | EPOLLERR;

    if (epoll_ctl (epollFD, EPOLL_CTL_ADD, socketFD, &newPeerConnectionEvent) == -1)
    {
       printf ("Could not add the socket FD to the epoll FD list. Aborting!");
       exit (2);
    }

    numEvents = epoll_wait (epollFD, &processableEvents, 1, -1);

    if (numEvents < 0)
    {
       printf ("Serious error in epoll setup: epoll_wait () returned < 0 status!");
       exit (2);
    }

    if (getsockopt (socketFD, SOL_SOCKET, SO_ERROR, &retVal, &retValLen) < 0)
    {
       // ERROR, fail somehow, close socket
    }

    if (retVal != 0) 
    {
       // ERROR: connect did not "go through"
    }   
}
else
{
   // ERROR: connect did not "go through" for other non-recoverable reasons.
   switch (errno)
   {
     ...
   }
}

I believe your error check after the epoll_wait() is incorrect - you should always check the result of the connection attempt via getsockopt(SO_ERROR), even if you didn't get EPOLLERR. See EINPROGRESS in the man page http://linux.die.net/man/2/connect Also, assert() is the wrong way to handle critical errors - it would mean that you have *proven* that it can never happen. Use exit() instead, which will terminate the program even when NDEBUG is defined. — Ambroz Bizjak, Jun 05 '12 at 21:47
Just added the edits suggested. The un-edited version seems to work for me. — Sonny, Jun 06 '12 at 01:20
Adding -1 as the timeout, shouldn't the epoll_wait block indefinitely in the above program? — wonder, Nov 30 '19 at 18:26

Aaron · Answer 4 · 2014-05-03T07:52:42.383

1

I have tried the Sonny's solution and the epoll_ctl will return invalid argument. So i think maybe the right way to do this is as follow:

1.create socketfd and epollfd

2.use epoll_ctl to associate the socketfd and epollfd with epoll event.

3.do connect(socketfd,...)

4.check the return value or errno

5.if errno == EINPROGRESS, do epoll_wait

edited May 03 '14 at 07:52

answered May 03 '14 at 07:46

Aaron

11
2

Async connect and disconnect with epoll (Linux)

4 Answers4

Linked