1

I am building a load generator for a simple Python multi-threaded web server. The load generator takes a user-provided number of threads, starts up a client, connects to server and requests a short file, then disconnects. The server creates a new thread for every new client connection.

What is happening is I am able to instantiate about 100 client threads without any issue. But When I increase it to 1000 or more threads, I get undesirable results. I get the following errors:

  • error: [Errno 32] Broken pipe when the client calls sendall,
  • error: [Errno 54] Connection reset by peer when the client calls recv.

What is going on and what is the proper way to handle these errors?

Edit
Clearly, I can just catch these socket errors. Is it appropriate to just catch the above errors, or should they be handled? If they should be handled, how should they be handled? These errors are not infrequent or randomly occurring; they occur consistently whenever my load generator spawns > ~100 threads.

I understand now that [Errno 32] Broken pipe occurs when the receiving side of the socket has been closed, and then the write side tries to write to it. In that case, then the receive side (server-side) is closing prematurely, but I'm not sure how to prevent this from happening or if there is even a way.

cosmosa
  • 743
  • 1
  • 10
  • 23
  • 2
    not sure thou, but they're all related clients disconnecting, you should simply catch those errors as exceptions and close the connection. http://stackoverflow.com/questions/180095/how-to-handle-a-broken-pipe-sigpipe-in-python – HassenPy Apr 26 '15 at 16:24
  • In CPython it is not a given that using threads improves performance because of the GIL. (You won't be using multiple cores simultaneously when running Python bytecode in multiple threads.) If you use `multiprocessing` you can actually *use* all cores. – Roland Smith Apr 26 '15 at 20:50

1 Answers1

2

That might actually reflect the load limits of the simple multi-threaded web server in question.

CPython, the standard Python implementation, doesn't use multiple cores for multithreading because of it's global interpreter lock (GIL). So compute-bound bound tasks like answering loads of HTTP requests are not going to fare so well.

To improve parallel performance, you can:

  • use multiprocessing, like multiprocessing,
  • use alternative Python implementation without such threading limitations, like Jython,
  • use some alternative approach to concurrency, e.g. microthreads like Greenlet, event loop like Twisted, etc.

Production grade web servers usually make use of both options, using multiple worker processes each running multiple threads.

famousgarkin
  • 13,687
  • 5
  • 58
  • 74
  • What I'd really like is a solution short of switching frameworks or Python implementations. I feel like the way I'm implementing the sockets is just incorrect. Maybe identifying why these errors are occurring would help me improve my implementation. – cosmosa Apr 26 '15 at 19:53
  • @cosmos1990 Might be the case as well. Just gave some pointers. You can't evade this debate when *threading with Python* :) – famousgarkin Apr 26 '15 at 20:08
  • 1
    Testing web servers is almost as big a debate as threading in cpython. :-) At the moment it seems that a multiprocessing event-driven design like nginx is [doing quite well](http://wiki.dreamhost.com/Web_Server_Performance_Comparison) for serving static content. – Roland Smith Apr 26 '15 at 21:02
  • 1
    @cosmos1990 If you want to know *why* your implementation tops out at 500 threads/second, you should *measure*. My guess would be that the overhead of thread creation and cleanup is limiting you. *Use a profiler* to see where the program is spending most of its time. However, often the best way to deal with sluggish performance is to implement a different algorithm/strategy. – Roland Smith Apr 26 '15 at 21:09
  • Sure, profiling is a good idea. But I am interested in solutions for handling the above errors, and understanding why they are occurring. – cosmosa Apr 28 '15 at 18:17