Django - End of script output before headers

Question

Basic hello.wsgi works fine. Other django application also works perfect with absolutely the same virtualenv version.

mod_wsgi hangs for ~10 min (!!!), and i see nothing can point me a problem (loglevel debug):

[Tue Mar 25 23:11:08.878578 2014] [:info] [pid 4719:tid 140720591648640] mod_wsgi (pid=4719): Attach interpreter ''.
[Tue Mar 25 23:11:08.879171 2014] [:info] [pid 4717:tid 140720591648640] mod_wsgi (pid=4717): Attach interpreter ''.
[Tue Mar 25 23:11:10.891348 2014] [authz_core:debug] [pid 4730:tid 140720477226752] mod_authz_core.c(802): [client 127.0.0.1:40844] AH01626: authorization result of Require all granted: granted
[Tue Mar 25 23:11:10.891391 2014] [authz_core:debug] [pid 4730:tid 140720477226752] mod_authz_core.c(802): [client 127.0.0.1:40844] AH01626: authorization result of <RequireAny>: granted
[Tue Mar 25 23:11:10.891489 2014] [authz_core:debug] [pid 4730:tid 140720477226752] mod_authz_core.c(802): [client 127.0.0.1:40844] AH01626: authorization result of Require all granted: granted
[Tue Mar 25 23:11:10.891497 2014] [authz_core:debug] [pid 4730:tid 140720477226752] mod_authz_core.c(802): [client 127.0.0.1:40844] AH01626: authorization result of <RequireAny>: granted
[Tue Mar 25 23:11:10.903921 2014] [:info] [pid 4719:tid 140720469034752] mod_wsgi (pid=4719): Create interpreter 'prj-dev|'.
[Tue Mar 25 23:11:10.905378 2014] [:info] [pid 4719:tid 140720469034752] [remote 127.0.0.1:57616] mod_wsgi (pid=4719, process='prj.project-dev', application='prj-dev|'): Loading WSGI script '/var/www/prj.project-dev/venv/www/public/index.wsgi'.
[Tue Mar 25 23:11:12.014799 2014] [:error] [pid 4719:tid 140720469034752] /usr/lib/python2.7/dist-packages/numpy/oldnumeric/__init__.py:11: ModuleDeprecationWarning: The oldnumeric module will be dropped in Numpy 1.9
[Tue Mar 25 23:11:12.014843 2014] [:error] [pid 4719:tid 140720469034752]   warnings.warn(_msg, ModuleDeprecationWarning)
[Tue Mar 25 23:11:12.014846 2014] [:error] [pid 4719:tid 140720469034752] 
[Tue Mar 25 23:16:11.888631 2014] [:info] [pid 4719:tid 140720485820160] mod_wsgi (pid=4719): Daemon process deadlock timer expired, stopping process 'prj.project-dev'.
[Tue Mar 25 23:16:11.888761 2014] [:info] [pid 4719:tid 140720591648640] mod_wsgi (pid=4719): Shutdown requested 'prj.project-dev'.
[Tue Mar 25 23:16:16.889065 2014] [:info] [pid 4719:tid 140720121513728] mod_wsgi (pid=4719): Aborting process 'prj.project-dev'.
[Tue Mar 25 23:16:16.895499 2014] [core:error] [pid 4730:tid 140720477226752] [client 127.0.0.1:40844] End of script output before headers: index.wsgi
[Tue Mar 25 23:16:17.160980 2014] [:info] [pid 5295:tid 140720591648640] mod_wsgi (pid=5295): Attach interpreter ''.

Apache config also contains nothing interesting. The same configuration works on other server.

<VirtualHost *:80>
    ServerName prj-dev
    ServerAdmin admin@force.fm
    DocumentRoot "/var/www/prj.project-dev/venv/www/public"
    ErrorLog /var/log/apache2/prj.project-dev/error_log
    CustomLog /var/log/apache2/prj.project-dev/access_log common

    Alias /media/ /var/www/prj.project-dev/venv/prj/prj/media/
    Alias /static/ /var/www/prj.project-dev/venv/prj/prj/static/
    Alias /usermedia/ /var/www/prj.project-dev/venv/prj/prj/usermedia/

    <Directory /var/www/prj.project-dev/venv/prj/prj/usermedia>
        Order deny,allow
        Allow from all
    </Directory>

    <Directory /var/www/prj.project-dev/venv/prj/prj/media>
        Order deny,allow
        Allow from all
    </Directory>

    <Directory /var/www/prj.project-dev/venv/prj/prj/static>
        Order deny,allow
        Allow from all
    </Directory>

    WSGIDaemonProcess prj.project-dev user=user group=user processes=2 threads=2 display-name=%{GROUP}
    WSGIProcessGroup prj.project-dev
    WSGIScriptAlias / /var/www/prj.project-dev/venv/www/public/index.wsgi

    XSendFile on

    <Directory /var/www/prj.project-dev/venv/www/public>
        Order deny,allow
        Allow from all
    </Directory>

</VirtualHost>

Nothing interesting in wsgi:

import os, sys

activate_this = '/var/www/prj.project-dev/venv/bin/activate_this.py'
execfile(activate_this, dict(__file__=activate_this))
sys.path = [
    '/var/www/prj.project-dev/venv/lib/python2.7/site-packages/',
    '/var/www/prj.project-dev/venv/prj/',
    '/var/www/prj.project-dev/venv/',
] + sys.path

os.environ['DJANGO_SETTINGS_MODULE'] = 'prj.settings'

import django.core.handlers.wsgi
#from raven.contrib.django.middleware.wsgi import Sentry

application = django.core.handlers.wsgi.WSGIHandler()
#application = Sentry(django.core.handlers.wsgi.WSGIHandler())

My question is: how can i get the real reason it hangs?

Create a separate question and list the modules you are using, as output with ``pip feeze`` and likely can readily identify the problem ones. — Graham Dumpleton, Mar 13 '17 at 02:21

score 11 · Answer 1 · answered Mar 26 '14 at 02:06

11

This is covered by the mod_wsgi documentation.

http://code.google.com/p/modwsgi/wiki/ApplicationIssues#Python_Simplified_GIL_State_API

In short, you are likely using a Python extension module which is not implemented to work properly in a Python sub interpreter. That or the extension module is not releasing the GIL properly when doing blocking operations. Both can result in an interpreter deadlock.

Use the solution in the documentation and see if that helps.

answered Mar 26 '14 at 02:06

Graham Dumpleton

57,726
6
119
134

Thank you for your answer. I accidentally found why it happens while cleaning imports in project: in one module i imported a local lib with current app prefix (it's in `sys.path` too): `app.permissions`. Then i changed it to `permissions`, and it worked. It looks like a circular import, but there were no these problems with previous version of `apache 2.2.2 && mod_wsgi`. Also there were not this bug with django runserver. And still i have no idea how to catch it without accidents. May be use profiler wrapper around application to see most hard calls? But will it produce output if wsgi kill it? – night-crawler Mar 26 '14 at 14:41
That doesn't sound right. Circular imports should not cause deadlocks. – Graham Dumpleton Mar 28 '14 at 00:34
3

I know experience this problem with matplotlib. Matplotlib's "print_figure" reproducably hangs with the above timeout error message. `WSGIApplicationGroup %{GLOBAL}` helps. Everything with Ubuntu 14.10's standard packages. Known problem? – Torsten Bronger May 08 '15 at 12:29
Yes, matplotlib is one package known to have this problem. – Graham Dumpleton May 08 '15 at 22:43
@TorstenBronger you should make an answer out of that – User Mar 17 '17 at 06:08
Thank you much for this! Is there any resource that compiles a list of packages with known GIL issues like this? I'm running into a similar problem with a Flask app, despite having WSGIApplicationGroup %{GLOBAL} as described in the link. – Sean Easter Jul 24 '17 at 15:56
If you are having what appears to be a similar issue, ask a new question about it and explain it plus provide your mod_wsgi configuration. It is not uncommon to have people get their configuration wrong such that ``WSGIApplicationGroup %{GLOBAL} `` isn't being used as they think. – Graham Dumpleton Jul 24 '17 at 21:01

score 3 · Answer 2 · edited Mar 13 '17 at 02:07

<VirtualHost *:80>
    ServerName localhost
    ServerAlias localhost
    ServerAdmin raghav@xyz.com

    <Directory /var/www/mysite/mysite>
    AddHandler wsgi-script .py
    Options +ExecCGI
    </Directory>

    DocumentRoot /var/www/mysite
    WSGIDaemonProcess myproject python-path=/var/www/mysite:/var/www/environ/lib/python2.7/site-packages
    #WSGIProcessGroup mysite
    WSGIScriptAlias / /var/www/mysite/mysite/wsgi.py

    WSGIApplicationGroup %{GLOBAL}

    <Directory /var/www/mysite/mysite/>


    order deny,allow
    Allow from all
    </Directory>

I also had the same error and I have solved this issue by using this configuration.

Also add the following line to /etc/apache2/apache.config

#WSGIPythonPath /var/www/mysite

score 3 · Answer 3 · answered May 20 '17 at 13:06

I am aware this thread is three years old but I stumbled across it when I had a similar issue so I thought I would add my solution. In my case it was something in the virtual environment that required mod_ssl to be enabled in Apache even though none of the sites used SSL (well they do but it is terminated elsewhere). So

a2enmod ssl
service apache2 restart

Fixed it.

score 1 · Answer 4 · answered May 31 '17 at 03:08

1

After digging through the answers above. I had to set the number of processes from 5 to 1. Also using the multi worker mpm.

LoadModule mpm_worker_module modules/mod_mpm_worker.so
WSGIDaemonProcess music user=apache group=apache processes=1 threads=30  python-path=/srv/www/music/service/venv/lib/python2.7/site-packages

answered May 31 '17 at 03:08

Pat W

36
2

That in itself shouldn't make a difference, it is forcing ``WSGIApplicationGroup %{GLOBAL}`` which is nearly always the answer to this problem. Also read up http://modwsgi.readthedocs.io/en/develop/user-guides/virtual-environments.html about setting the virtual environment in preferred way. Don't use ``python-path`` to do it. – Graham Dumpleton May 31 '17 at 07:04

Django - End of script output before headers

4 Answers4

Linked