24

I'm trying to deploy a Django application to Elastic Beanstalk. When I visit the page it never loads. The logs say:

Script timed out before returning headers: wsgi.py

I can ssh into the server and run manage.py runserver and then curl 127.0.0.1:8000 from another terminal, which will return the page successfully. So I'm assuming it must be an issue with the Apache configuration that is set up as a part of Elastic Beanstalk.

Here is more of the logs:

[pid 15880] AH01232: suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[so:warn] [pid 15880] AH01574: module wsgi_module is already loaded, skipping
[auth_digest:notice] [pid 15880] AH01757: generating secret for digest authentication ...
[lbmethod_heartbeat:notice] [pid 15880] AH02282: No slotmem from mod_heartmonitor
[mpm_prefork:notice] [pid 15880] AH00163: Apache/2.4.9 (Amazon) mod_wsgi/3.4 Python/2.7.5       configured -- resuming normal operations
[core:notice] [pid 15880] AH00094: Command line: '/usr/sbin/httpd -D FOREGROUND'
[:error] [pid 15881] /opt/python/run/venv/lib/python2.7/site-packages/numpy/oldnumeric/__init__.py:11: ModuleDeprecationWarning: The oldnumeric module will be dropped in Numpy 1.9
[:error] [pid 15881]   warnings.warn(_msg, ModuleDeprecationWarning)
[:error] [pid 15881] 
[core:error] [pid 15884] [client 10.248.110.45:58996] Script timed out before returning headers: wsgi.py

And my wsgi.py file:

import os
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "aurora.settings")

from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()

Any clues as to what could be causing this?

UPDATE:

I rebuilt my environment and ran into this issue again. I updated /etc/httpd/conf.d/wsgi.conf to include WSGIApplicationGroup %{GLOBAL} as mentioned here. I am using Scipy, Numpy, and GeoDjango (which uses GDAL). I know GDAL is not entirely thread safe and I'm not sure about the others but I'm assuming it was a thread safety issue.

Meistro
  • 3,664
  • 2
  • 28
  • 33
  • 4
    I'm using pandas and numpy and got the same issue. Adding `WSGIApplicationGroup %{GLOBAL}` solved the issue - however manualy editing the file doesn't fix it for new beanstalk instances. To solve that, follow steps described in [this answer](http://stackoverflow.com/questions/41812497/aws-elastic-beanstalk-script-timed-out-before-returning-headers-application-p/41855346#41855346) – David Viktora Feb 08 '17 at 19:49
  • same here after rebooting the beanstalk instance. – Mattia Paterna Aug 10 '17 at 13:39

6 Answers6

14

UPDATE 8 FEB 2017

Previously my wsgi.conf was only using one process:

WSGIDaemonProcess wsgi processes=1 threads=15 display-name=%{GROUP}

I upped the processes to something more reasonable and haven't had any issues:

WSGIDaemonProcess wsgi processes=6 threads=15 display-name=%{GROUP}

This change along with the original addition of WSGIApplicationGroup %{GLOBAL} seems to have done the trick.

UPDATE 17 September 2015

I'm still occasionally running in to this issue. Usually, redeploying via eb deploy fixes the issue. It's hard to say what the underlying issue is.

Original Answer

I eventually got the project working but then tried creating an image to use for new instances, which reopened the problem. I'm not sure why it worked then stopped working but I rebuilt my custom AMI from scratch and then repushed my project. Turns out it was an issue in wsgi.py. The version I posted was actually the different from what was being deployed. For some reason another developer had put this in wsgi.py:

path = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
if path not in sys.path:
sys.path.append(path)

I removed this and it fixed the problem.

My advice for anyone having 

Script timed out before returning headers: wsgi.py

is to check you wsgi.py file.

Meistro
  • 3,664
  • 2
  • 28
  • 33
  • Are you still facing this problem? I am made a deploy of two application on eb. One of them worked well while the other (that I is using scipy) is facing this problem. – silviomoreto Feb 16 '16 at 16:03
  • where does wsgi.py reside? – Jun Sep 27 '18 at 19:20
  • I can confirm that this issue has to do with installation of `numpy` and many other elements of the Python scientific stack on recent versions of `pip`. I have encountered it multiple times after upgrading `numpy` on older Django servers. – Chris Conlan Jul 20 '20 at 16:02
5

The fix for us was to add the WSGIApplicationGroup %{GLOBAL} setting as per Meistro's answer.

You'll want to ensure you edit your wsgi config by way of your .ebextensions/foobar.config file, so that the changes are permanent. See the .ebextensions config docs.

Add the following to your .ebextensions/foobar.config file:

files:
  "/etc/httpd/conf.d/wsgi_custom.conf":
    mode: "000644"
    owner: root
    group: root
    content: |
      WSGIApplicationGroup %{GLOBAL}

This will create (or overwrite) the contents of the /etc/httpd/conf.d/wsgi_custom.conf file with WSGIApplicationGroup %{GLOBAL}

Community
  • 1
  • 1
Chris W.
  • 37,583
  • 36
  • 99
  • 136
4

It certainly does seem like an issue with WSGI and Apache like you mentioned. One thing to double check is the .ebextensions file in your source directory.

There should be a config in there that specifies the WSGI information like the location of the application. You might also want to check your Django settings and run it locally with an Apache setup using WSGI.

You've probably already read the official documentation for WSGI and Django, but this might catch some simplistic things that you might have missed: http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/create_deploy_Python_django.html#create_deploy_Python_django_update

Josh Davis
  • 6,651
  • 2
  • 25
  • 25
  • Thanks for the answer Josh. WSGI location is correct and I have read the docs. For some more context I'm using a custom AMI and I briefly had it working, but then I immediately tried to create an image, which broke the whole thing again. I'm not sure if you have any tips, since you work on EBS, but installing anything not trivial like Scipy and Numpy is extremely painful since `pip install scipy` takes a long time and will eventually timeout. But baking a custom AMI with all of the pip packages seems impossible as well. – Meistro Aug 25 '14 at 05:31
  • I'm glad you were able to figure it out. Cheers. – Josh Davis Aug 25 '14 at 22:09
  • 1
    Thanks for looking in to it. Confirming it was an Apache/WSGI issue helped me focus in. Best. – Meistro Aug 25 '14 at 22:28
4

Just my 2 cents on a similar issue I faced.

I had a similar problem. It turned out to be that the script (which makes a DB insert/update/delete call) being executed from django application was waiting in a deadlock state for the lock on the table to be released. Once it was released, the code went through without this error. I never had to re-deploy my EB application.

  1. Make sure that you are not connected to the database through any other SQL client. If yes, query for any active locks. (In my case, I was connecting to redshift. Querying STV_LOCKS table lists the active locks).
  2. Check who is holding the locks. (In my case, it was my SQL client, connected to redshift, which executed a CUD operation, and since COMMIT was not issued, it was holding a pending write lock on the table).
  3. I issued a commit from my SQL client, and the lock was released. My EB code went through and there was no Script timed out error any more.
Kris
  • 51
  • 4
1

I had this problem until I realized that I was using a different Python version. Let me explain this. I was using a CentOS 7. The default Python version in CentOS 7 is Python 2.7, but my code use Python 3.6, so I installed Python 3.6 in the same machine by doing this:

sudo yum install centos-release-scl
sudo yum install rh-python36 rh-python36-python-pip rh-python36-python-virtualenv
scl enable rh-python36 bash

Then created a virtual environment and used it in WSGI:

    WSGIScriptAlias   / /var/www/myproject/myproject/wsgi.py
    WSGIDaemonProcess myproject python-home=/var/www/myproject python-path=/var/www/myproject:/var/www/myproject/lib/python3.6/site-packages
    WSGIProcessGroup  myproject

And the "script timed out" problem appeared. Then I realize that the mod_wsgi.so was compiled against libpython2.7.so.1.0.

# ldd /usr/lib64/httpd/modules/mod_wsgi.so
linux-vdso.so.1 =>  (0x00007ffd3fcae000)
libpython2.7.so.1.0 => /lib64/libpython2.7.so.1.0 (0x00007f1240cd4000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f1240ab8000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f12408b3000)
libutil.so.1 => /lib64/libutil.so.1 (0x00007f12406b0000)
libm.so.6 => /lib64/libm.so.6 (0x00007f12403ae000)
libc.so.6 => /lib64/libc.so.6 (0x00007f123ffe0000)
/lib64/ld-linux-x86-64.so.2 (0x00005598a5aac000)

The solution was removing mod_wsgi package and installing rh-python36-mod_wsgi package.

Best regards!

Bstia
  • 11
  • 1
0

I have tried above steps which can solve issue temporally. then I have done bellow steps:

  1. make "packages.config" file under ".ebextensions " folder and put following lines

    WSGIApplicationGroup: command: grep -rnw 'WSGIApplicationGroup' config.py || sed -i.bak '/LogFormat/ a WSGIApplicationGroup %%{GLOBAL}' config.py cwd: /opt/elasticbeanstalk/hooks

thanks for helps to made this who given suggestion for this type of error

I finally fixed. just read about apache mpm load module concept

I was changed the default load mode from apache preforker(my case) module which is depends on os.

find below location

location : /etc/httpd/conf.module.d/00-mpm*.

Enable worker module which depends our cases

LoadModule mpm_worker_module lib64/httpd/modules/mod_mpm_worker.so

once again thanks for who helped me.

selvasundarraj
  • 21
  • 3