1

I'm confused whether using multiple processes for a web application will improve the performance. Apache's mod_wsgi provides an option to set the number of processes to be started for the daemon process group. I used fastcgi with lighttpd before and it also had an option to configure the max number of processes for each fastcgi application.

While I don't know how multi-processing is better, I do know something bad about it compared to single-process multi-threading model. For example, logging will be harder to implement in multi-processing scenario (link), especially when you also want log rotating. And since memory can't be shared, if you cache something in memory (the most straightforward way), you have multiple duplicate copies.

Do multiple processes better utilize multi-core computing power, or do they yield higher throughput? Or are they just there for some single threaded applications?

Community
  • 1
  • 1
qingbo
  • 2,130
  • 16
  • 19

2 Answers2

4

In the case of Python, or more specifically CPython as used by mod_wsgi, the issue is the Python GIL. Although you may have multiple threads in Python, the global interpreter lock effectively means that only one thread can be running Python code at a time. This means it cannot make use of multiple processors/cores properly on a system. Using multiple processes however does allow you to use all those processors/cores.

That said, for mod_wsgi it isn't all Python code but has a lot of C code and with Apache also being C code. During execution of the C code, the GIL is unlocked by that thread meaning that a thread running in C code can run in parallel to a thread running in Python code. Still not the best one can achieve, but can still make partial use of all the processors/core on your system.

For me details of this in relation to mod_wsgi read:

http://blog.dscpl.com.au/2007/09/parallel-python-discussion-and-modwsgi.html http://blog.dscpl.com.au/2007/07/web-hosting-landscape-and-modwsgi.html

Graham Dumpleton
  • 57,726
  • 6
  • 119
  • 134
3

Multi-processing is less efficient than multi-threading, but it's more resilient to failure. Each process gets its own independent memory space and may be terminated and restarted (recycled) independent of other processes.

Stephen Cleary
  • 437,863
  • 77
  • 675
  • 810
  • Its efficiency really depends on what you're trying to do, and under which OS. It's not something you can make such a blanket statement on. – Cory Nelson Aug 09 '11 at 17:26
  • @Cory: While his statements about efficiency could be wrong in theory, on the OSes under which Apache is normally used (e.g., Linux, *BSD, Windows, OS/X) they're accurate enough to be meaningful/useful. – Jerry Coffin Aug 09 '11 at 19:14
  • @Cory, this answer isn't about efficiency; it's about safety. – MSN Aug 10 '11 at 00:20
  • @MSN He mentions efficiency. So yes, it is. I was not arguing against or minimizing the other parts of his answer. – Cory Nelson Aug 10 '11 at 02:01