6

I'm trying to get a Flask app running on AWS EC2 (standard Ubuntu AMI) through an Apache2 Webserver. My app internally uses Numpy. I have tested the following:

  1. Without Numpy, the App runs through Apache2. With Numpy, it trips on the import numpy and throws a 500 Server error (See logs below).

  2. With Numpy, the app runs fine when invoked directly from the command line (i.e. python app.py). The numpy bits work as expected and I can query app endpoints externally.

I also printed the system path (print sys.path) just before the import numpy as np and made sure that the appropriate site-packages directory is in the path. I also manually specified the WSGIPythonPath to include the site-packages directory. I tried setting the number of threads to 1 and editing the apache2.conf file. None of these efforts have been successful.

Below is my directory structure and contents of the relevant files.

Root folder /var/www/html/webserver_mockup

Root folder contents

/var/www/html/webserver_mockup/app.wsgi /var/www/html/webserver_mockup/mockup/__init__.py /var/www/html/webserver_mockup/mockup/app.py

app.wsgi

(Edited according to Graham's comment below)

#!/usr/bin/python
import sys
import site
site.addsitedir('/home/ubuntu/.local/lib/python2.7/site-packages')
sys.path.insert(0, "/var/www/html/webserver_mockup")

from mockup.app import app as application

/etc/apache2/sites-available/000-default.conf

(Edited according to Graham's comment below)

WSGIPythonPath /usr/local/lib/python2.7/site-packages/

<VirtualHost *:80>
    ServerAdmin webmaster@localhost
    DocumentRoot /var/www/html

    WSGIDaemonProcess webserver_mockup threads=5
    WSGIScriptAlias / /var/www/html/webserver_mockup/app.wsgi

    <Directory />
        WSGIProcessGroup %{GLOBAL}
        WSGIApplicationGroup %{GLOBAL}
        Order allow,deny
        Allow from all
    </Directory>
    LogLevel warn
    ErrorLog ${APACHE_LOG_DIR}/error.log
    CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>

Apache error log

[Fri Apr 07 04:09:41.062282 2017] [mpm_event:notice] [pid 7221:tid 
140325391972224] AH00489: Apache/2.4.18 (Ubuntu) mod_wsgi/4.3.0 
Python/2.7.12 configured -- resuming normal operations
[Fri Apr 07 04:09:41.062358 2017] [core:notice] [pid 7221:tid 
140325391972224] AH00094: Command line: '/usr/sbin/apache2'
[Fri Apr 07 04:09:46.770159 2017] [wsgi:error] [pid 7225:tid 
140325250004736] [client 24.6.50.183:58332] mod_wsgi (pid=7225): Target 
WSGI script '/var/www/html/webserver_mockup/app.wsgi' cannot 
be loaded as Python module.
[Fri Apr 07 04:09:46.770300 2017] [wsgi:error] [pid 7225:tid 
140325250004736] [client 24.6.50.183:58332] mod_wsgi (pid=7225): 
Exception occurred processing WSGI script 
'/var/www/html/webserver_mockup/app.wsgi'.
[Fri Apr 07 04:09:46.770370 2017] [wsgi:error] [pid 7225:tid 
 140325250004736] [client 24.6.50.183:58332] Traceback (most recent 
call last):
[Fri Apr 07 04:09:46.770432 2017] [wsgi:error] [pid 7225:tid 
140325250004736] [client 24.6.50.183:58332]   File 
"/var/www/html/webserver_mockup/app.wsgi", line 7, in <module>
[Fri Apr 07 04:09:46.770534 2017] [wsgi:error] [pid 7225:tid 
140325250004736] [client 24.6.50.183:58332]     from mockup.app import 
app as application
[Fri Apr 07 04:09:46.770612 2017] [wsgi:error] [pid 7225:tid 
140325250004736] [client 24.6.50.183:58332]   File 
"/var/www/html/webserver_mockup/mockup/app.py", line 12, in 
<module>
[Fri Apr 07 04:09:46.770693 2017] [wsgi:error] [pid 7225:tid 
140325250004736] [client 24.6.50.183:58332]     import numpy as np
[Fri Apr 07 04:09:46.770755 2017] [wsgi:error] [pid 7225:tid 
140325250004736] [client 24.6.50.183:58332] ImportError: No module 
named numpy

My question is: Why is Apache2 not finding Numpy despite the appropriate site-packages being in its path?

RDK
  • 923
  • 1
  • 9
  • 22

2 Answers2

5

This:

site.addsitedir('/home/ubuntu/.local/lib/python2.7/site-packages/numpy')

should have been:

site.addsitedir('/home/ubuntu/.local/lib/python2.7/site-packages')

But the whole way you are setting up Python virtual environments is not best practice anyway. Suggest you read:

Also try and switch to daemon mode, it is better to use daemon mode than embedded mode.

Graham Dumpleton
  • 57,726
  • 6
  • 119
  • 134
  • Thank you Graham for your prompt answer to the question. This is my first time setting up a Numpy-based application within `mod-wsgi`. Per your suggestion, I've changed the call to `addsitedir()` and `deamon`ized the process. The issue is still not solved. The modified `000-default.conf` appears in the above question. – RDK Apr 07 '17 at 17:01
  • Also, I'm not using Virtualenvironment. Are you suggesting that Virtualenvironment should be used? – RDK Apr 07 '17 at 17:07
  • 1
    The other possible reason is that the user Apache runs your code as cannot see into your home directory. Usually permissions on home directories don't allow others to see inside of it. – Graham Dumpleton Apr 07 '17 at 20:32
  • Hi Graham, thanks for your comment. I'd figured out the reason, but couldn't add a comment to that effect. If you add this to your answer, I'll be glad to accept it. – RDK Apr 14 '17 at 06:59
2

numpy via pip is not installed at /home/ubuntu/.local/lib/python2.7/site-packages but it should be at /home/ubuntu/.local/lib64/python2.7/site-packages or something like that.

Just try add another site.addsitedir in your app.wsgi to that folder should work. Something like this :

site.addsitedir('/home/ubuntu/.local/lib64/python2.7/site-packages')

Dat TT
  • 2,850
  • 2
  • 16
  • 18