BasicHTTPServer, SimpleHTTPServer and concurrency

Question

I'm writing a small web server for testing purposes using python, BasicHTTPServer and SimpleHTTPServer. It looks like it's processing one request at a time. Is there any way to make it a little faster without messing around too deeply? Basicly my code looks as the following and I'd like to keep it this simple ;)

os.chdir(webroot)
httpd = BaseHTTPServer.HTTPServer(("", port), SimpleHTTPServer.SimpleHTTPRequestHandler)
print("Serving directory %s on port %i" %(webroot, port) )
try:
 httpd.serve_forever()
except KeyboardInterrupt:
 print("Server stopped.")

why dont you use twisted for creating a HTTPServer? – anijhaw Mar 16 '10 at 17:06 — anijhaw, Mar 16 '10 at 17:06

score 9 · Accepted Answer · edited May 23 '17 at 11:46

9

You can make your own threading or forking class with a mixin inheritance from SocketServer:

import SocketServer
import BaseHTTPServer

class ThreadingHTTPServer(SocketServer.ThreadingMixIn, BaseHTTPServer.HTTPServer):
    pass

This has its limits as it doesn't use a thread pool, is limited by the GIT, etc, but it could help a little (with relatively little effort). Remember that requests will be served simultaneously by multiple threads, so be sure to put proper locking around accesses to global/shared data (unless such data's immutable after startup) done in the course of serving a request.

This SO question covers the same ground (not particularly at length).

edited May 23 '17 at 11:46

Community

1
1

answered Mar 16 '10 at 15:39

Alex Martelli

854,459
170
1,222
1,395

Sadly I'm stuck on 16 requests per second - with 1 concurrent request as well as with 10. – braindump Mar 16 '10 at 16:23
2

@braindump, if you're on any form of Unix (basically anything except Windows), try a ForkingMixin instead of the ThreadingMixin, it might serve you better, esp. if you're on a multicore CPU. Otherwise, you'll really need to move to some kind of "real" server (Tornado, for example: see http://www.tornadoweb.org/ -- simple, and very high performance per http://bret.appspot.com/entry/tornado-web-server ). – Alex Martelli Mar 16 '10 at 17:33
I'm on Gentoo Linux on an ARM-based CPU. Thanks for your hint regarding Forking, it's much better now, even if it's not good ;) But for my intention, it's okay. (On 10 concurrent requests, I got 60 requests per second. Lighttpd reaches ~400 requests per second on the same machine) – braindump Mar 16 '10 at 19:02
@braindump, lighttpd's based on an async approach, I believe, just like Tornado, which I recommended. But if 60 QPS are OK for your purposes, there's no real need for you to check if Tornado would reach 300, 400, or whatever;-). – Alex Martelli Mar 16 '10 at 20:43
Thank for useful tip! A tiny import/typo fixup, should be "from SocketServer import ThreadingMixIn" (not imported as submodule otherwise, at least under python 2.6) and later use "ThreadingMixIn" (also note the capital 'I' in MixIn). – Yonatan Jan 27 '11 at 15:18

score 1 · Answer 2 · answered Mar 16 '10 at 16:58

1

You might also look at CherryPy -- it's pretty simple, too, and has multiple request threads with no additional effort. Although your needs may be modest now, CP has a lot of nice capabilities that may benefit you in the future.

answered Mar 16 '10 at 16:58

jgarbers

231
3
13

Are you sure about concurrency? I can't find it in their documentation. – iwtu Jun 11 '15 at 08:31

score 0 · Answer 3 · answered Mar 16 '10 at 15:49

0

Depending on what your requirements are, another option might be to hook in Paste. Though, based on your example, it may be overkill. Something to keep in the toolbox.

answered Mar 16 '10 at 15:49

McJeff

331
1
4

BasicHTTPServer, SimpleHTTPServer and concurrency

3 Answers3

Linked