Elastic Beanstalk installing scipy

Question

I'm trying to install scipy on a bare-bones Django app running on Amazon Elastic Beanstalk, but I can't get it to work.

Here are the steps to reproduce my problem:

The regular stuff from the guide:

# Create a new virtual environment
mkvirtualenv -p python2.7 django_eb

# Install Django on it
pip install django==1.9.12
# pip freeze should now show Django==1.9.12 and some other things

# Start a new Django project 
# This creates a directory that has everything you need for Django
django-admin startproject django_eb
cd django_eb

# Optionally check that the site works
python manage.py runserver
Ctrl-C

# Store the pip requirements so that the remote host can install them
pip freeze > requirements.txt

# Tell the remote server where our wsgi file is
mkdir .ebextensions
cat <<EOT >> .ebextensions/django.config
option_settings:
  aws:elasticbeanstalk:container:python:
    WSGIPath: django_eb/wsgi.py
EOT

# Allow any host for our project
# If this is unset, you'll get a 404 on the deployed site
set ALLOWED_HOSTS = ['*'] in settings.py

# Create an EB project
# Will need AWS EB CLI for this
eb init -p python2.7 django_eb
# Choose some region
eb init
# choose Y so we can SSH and check logs

# Create a deployment environment
eb create django-eb-env 
# This step takes around 5 minutes
# If it fails and you need to restart run 'eb deploy'

# Open the website in your OS's default browser
eb open
# If you get DisallowedHost at / error
# double check that ALLOWED_HOSTS = ['*']

Installing scipy:

# Now we'll install scipy and watch how it doesn't work remotely
pip install scipy==0.19.0
pip freeze > requirements.txt
eb deploy
# Should take forever and then finally print 
# 'ERROR: Failed to deploy application.'

eb ssh
cat /var/log/eb-activity.log
# Should print
# numpy.distutils.system_info.NotFoundError: no lapack/blas resources found

# After reading this blog post
# https://medium.com/@DaveJMcKeown/deploying-scipy-into-aws-elastic-beanstalk-2e5e481155de
# I added this to .ebextensions/django.config:
packages:  
  yum:
    make: []
    gcc-c++: []
    gcc-gfortran: []
    python27-devel: [] # I used python27-devel instead of python-devel
    atlas-sse3-devel: []
    lapack-devel: []
    libpng-devel: []
    freetype-devel: []
    zlib-devel: []
container_commands:  
  AddGlobalWSGIGroupAccess: 
    command: "if ! grep -q 'WSGIApplicationGroup %{GLOBAL}' ../wsgi.conf ; then echo 'WSGIApplicationGroup %{GLOBAL}' >> ../wsgi.conf; fi;"

# Unfortunately, this leads to an our of memory error
# Running dmesg now looks like the following:
# http://stackoverflow.com/a/35011967/2770572

I'm at a loss here. It seems like I could maybe get this to work on an EC2 instance with more RAM, but I can't really do that because it'll take me out of the free tier. Is there a way to run the make command so that it doesn't take up so much memory or some other solution?

Same here. I found this way to tell the EB to upload my pip version: `commands: 00_update_pip: command: "/opt/python/run/venv/bin/pip install --upgrade pip"`. I would love to know if there is any way to tell the EB something like `pip install --no-cache-dir -r requirements.txt`. I remember I had it several times when I was installing packages *directiy* on the EB instance, and I solved it this way. — Mattia Paterna, Aug 10 '17 at 11:45
@MattiaPaterna I stopped working on it by the time you answered so idk — michaelsnowden, Nov 21 '17 at 19:11

Mattia Paterna · Answer 1 · 2017-08-10T12:58:34.723

You can try to add this to your .ebextensions/django.config:

commands:
  01_no_cache:
    command: "/opt/python/run/venv/bin/pip install --no-cache-dir -r /opt/python/current/app/requirements.txt"

If you inspect /var/log/eb-activity.log (or just type eb logs if you have the EB CLI installed), you can see the error here:

MemoryError
2017-08-10 11:40:25,557 ERROR    Error installing dependencies: Command '/opt/python/run/venv/bin/pip install -r /opt/python/ondeck/app/requirements.txt' returned non-zero exit status 2

As far I can see, you can provide your EB with specific commands if you include them inside the django.config. As written here, you can disable the cache.

This has worked for me.

Elastic Beanstalk installing scipy

1 Answers1