2

I have a problem with my multithreaded networking server program.

I have a main thread that is listening for new client connections. I use Linux epoll to get I/O event notifications. For each incoming event, I create a thread that accept() the new connection and assign a fd to it. Under heavy loading, it can occur that the same fd is assigned twice causing my program to crash.

My question is: how can the system re-assign a fd that is still used by another thread?

Thanks,

  • possible duplicate of [C: epoll and multithreading](http://stackoverflow.com/questions/4687626/c-epoll-and-multithreading) – Nick Jun 12 '13 at 12:35

3 Answers3

3

Presumably there is a race condition here - but without seeing your code it's hard to diagnose.

You would be better to accept on the Main thread and then pass the accepted socket to the new thread.

If you pass your listening socket to a new thread to then perform the accept - you're going to hit a race condition.

For further information you can look here: https://stackoverflow.com/a/4687952/516138

And this is a good background on networking efficiency (although perhaps a bit out of date).

Community
  • 1
  • 1
Nick
  • 25,026
  • 7
  • 51
  • 83
  • Ok so based on what you said, I have two solutions to my problem. The first one, is to `accept` on the main thread then pass the assigned fd to the new worker thread. The second one, is to have a dedicated thread for each epoll loop. One of these two solutions might resolve my race condition issue. Is there any documentation somewhere to get more detailed description about epoll and concurrency? –  Jun 12 '13 at 11:42
  • A third option and the choice I'd make if I were writing a networked application in C++ from scratch would be to use [ASIO](http://think-async.com/) (or [Boost.Asio](http://www.boost.org/libs/asio)). – Nick Jun 12 '13 at 12:31
0

You should call accept() on the same thread that you are calling epoll() on. Otherwise you are inviting race conditions.

user207421
  • 305,947
  • 44
  • 307
  • 483
0

File descriptors are modified in a "per process basis". This means that they are unique for each process. This means that multiple threads can share the same file descriptors in the same process.

Having an accept syscall returning the same file descriptor inside the same process is a very strong indication that some of your threads are closing the previous "version" of the repeated file descriptor.

Issues like this one may be difficult to debug in complex software. A way to identify that in Linux system is to use the strace command. One can run strace -f -e trace=close,accept4,accept,pipe,open <your program>. That's going to output on your screen the respective syscalls specified in the command along with which thread is calling it.

Gustavo Meira
  • 2,875
  • 3
  • 21
  • 33