3

I have a simple fcgi server written in c++ - for now it's single threaded. I saw the main loop for example like

accept_connections();
handle_data();`
handle_connections();

but what if I want to respond to each client simultaneously? Because if I understand this, server first accepts new connections and iterate over them and proccess each. But this can be not good because of for example a upload (while one client uploads, others have to wait.) or by DoS like slowloris. How can server do multiple clients at one time ? I saw one server it was single-threaded - and did it.

How to do it ?

Zaffy
  • 16,801
  • 8
  • 50
  • 77
  • 2
    Single-threaded design depends a lot on asynchronous IO. Check your platform (BSD sockets?) for asynchronous read and write operations. – Kerrek SB Sep 10 '11 at 13:05

3 Answers3

2

I solved it earlier but i want to answer my question.

Solution I made:

  • 1 acceptor thread - (accept() for new incoming connections in blocking behavior) and assigning connections to network threads with load balancing
  • N network threads where N = (1 thread per 1000 clients) with nonblocking behavior:
    • initialize
    • while server is running
      • set timeout watcher (for disabling blocking in event loop)
      • run event loop
      • stop timeout watcher
      • add new sockets from acceptor thread
      • iterate over connections run Update(), try to flush output, check for close, etc. if Update() return -1, erase connection from list
    • end while
    • release and close all connections and free all allocated memory
    • return
  • Worker Threads with posix semaphores waiting for jobs (thread pool as management)
  • (Optional) Background Thread with low priority for scheduling and checking, deleting for example tokens, sessions, etc. Lots of sleeping.

I wrote custom fastcgi handler. I also used libev for network threads, buffered and direct output and automatic waking and disabling write event. Debugging with valgrind. Compiler clang.

Reactions to other answers:

I didnt want to use thread per connection because it has some cost in spawning and also if there are lots of threads, there is also a slowdown and higher memory usage.

And I didnt use boost asio because I dislike frameworks and asynchronous IO was not one that i was looking for.

Zaffy
  • 16,801
  • 8
  • 50
  • 77
2

Look into Boost.Asio for asynchronous I/O using a single thread.

Sam Miller
  • 23,808
  • 4
  • 67
  • 87
  • I don't preffer to use frameworks, but thanks I'll take a look. – Zaffy Sep 10 '11 at 14:45
  • If you prefer libraries over frameworks, you may want to look at libuv. It's the lightweight, cross platform network abstraction library which is used in Node.js. It's singlethreaded and really fast (on all platforms!). – Milan Sep 10 '11 at 15:35
  • What is better - Non-blocking sockets or posix aio ? – Zaffy Sep 10 '11 at 22:04
  • Look at this thread (the first answer in particular): http://stackoverflow.com/questions/87892/what-is-the-status-of-posix-asynchronous-i-o-aio – Milan Sep 11 '11 at 11:00
0

You could spawn a new thread for each incoming connection. To increase efficiency, use a thread pool to avoid the overhead of creating a new thread all the time.

There are alternative/better methods, but this one is quite simple and you can continue to use blocking sockets. Watch out for synchronization issues when accessing shared resources from your threads though!

Milan
  • 3,342
  • 3
  • 31
  • 40
  • I want to avoid thread per connection - and i want to avoid this thread pool because it can fill fast and other clients must wait – Zaffy Sep 10 '11 at 14:44