Can't import modules on EC2 - Flask App

Question

I am unable to import modules like NumPy, sklearn etc within the flask app on an EC2 Micro instance.

Within the /home/ubuntu directory, I have a directory flaskapp which contains:

flaskapp.py
flaskapp.wsgi

The code in flaskapp.py is:

import sys
import os
import shutil
import time
import traceback

from flask import Flask, request, jsonify

app = Flask(__name__)

@app.route('/predict')
def predict():
    from sklearn.externals import joblib
    import numpy as np
    clf = joblib.load("flaskapp/models/saved_model.pkl")
    print 'model loaded'
    if clf:
        try:
            prediction = list(clf.predict(np.array([['0', '100', '164', '150', '1']])))
            return jsonify({'prediction': prediction})
        except Exception, e:
            return "ERROR" + e
    else:
        print os.getcwd()
        return 'no model here'


if __name__ == '__main__':
    from sklearn.externals import joblib
    import numpy as np
    print "Imports Done"
    app.run(DEBUG=True)

The flaskapp.wsgi looks like this:

import sys
import site
site.addsitedir('/home/ubuntu/.local/lib/python2.7/site-packages')
sys.path.insert(0, '/var/www/html/flaskapp')
from flaskapp import app as application

According to the following question: Numpy ImportError when deploying Flask App using mod_wsgi/Apache2

I modified my /etc/apache2/sites-enabled/000-default.conf (and sites-available too) as:

WSGIPythonPath /usr/local/lib/python2.7/site-packages/
<VirtualHost *:80>

        ServerAdmin webmaster@localhost
        DocumentRoot /var/www/html


        WSGIDaemonProcess flaskapp threads=5
        WSGIScriptAlias / /var/www/html/flaskapp/flaskapp.wsgi

        <Directory flaskapp>
                WSGIProcessGroup flaskapp
                WSGIApplicationGroup %{GLOBAL}
                Order deny,allow
                Allow from all
        </Directory>

        ErrorLog ${APACHE_LOG_DIR}/error.log
        CustomLog ${APACHE_LOG_DIR}/access.log combined

</VirtualHost>

Going to the URL http://ec2-....compute-1.amazonaws.com/predict throws:

500 Internal Server Error

The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.

The error log displays the following:

[Thu Nov 02 22:15:21.936872 2017] [wsgi:error] [pid 11753:tid 140263731681024] [client 130.245.192.4:11592] [2017-11-02 22:15:21,936] ERROR in app: Exception on /predict [GET]
[Thu Nov 02 22:15:21.936900 2017] [wsgi:error] [pid 11753:tid 140263731681024] [client 130.245.192.4:11592] Traceback (most recent call last):
[Thu Nov 02 22:15:21.936903 2017] [wsgi:error] [pid 11753:tid 140263731681024] [client 130.245.192.4:11592]   File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1982, in wsgi_app
[Thu Nov 02 22:15:21.936905 2017] [wsgi:error] [pid 11753:tid 140263731681024] [client 130.245.192.4:11592]     response = self.full_dispatch_request()
[Thu Nov 02 22:15:21.936907 2017] [wsgi:error] [pid 11753:tid 140263731681024] [client 130.245.192.4:11592]   File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1614, in full_dispatch_request
[Thu Nov 02 22:15:21.936909 2017] [wsgi:error] [pid 11753:tid 140263731681024] [client 130.245.192.4:11592]     rv = self.handle_user_exception(e)
[Thu Nov 02 22:15:21.936911 2017] [wsgi:error] [pid 11753:tid 140263731681024] [client 130.245.192.4:11592]   File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1517, in handle_user_exception
[Thu Nov 02 22:15:21.936913 2017] [wsgi:error] [pid 11753:tid 140263731681024] [client 130.245.192.4:11592]     reraise(exc_type, exc_value, tb)
[Thu Nov 02 22:15:21.936915 2017] [wsgi:error] [pid 11753:tid 140263731681024] [client 130.245.192.4:11592]   File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1612, in full_dispatch_request
[Thu Nov 02 22:15:21.936917 2017] [wsgi:error] [pid 11753:tid 140263731681024] [client 130.245.192.4:11592]     rv = self.dispatch_request()
[Thu Nov 02 22:15:21.936918 2017] [wsgi:error] [pid 11753:tid 140263731681024] [client 130.245.192.4:11592]   File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1598, in dispatch_request
[Thu Nov 02 22:15:21.936920 2017] [wsgi:error] [pid 11753:tid 140263731681024] [client 130.245.192.4:11592]     return self.view_functions[rule.endpoint](**req.view_args)
[Thu Nov 02 22:15:21.936922 2017] [wsgi:error] [pid 11753:tid 140263731681024] [client 130.245.192.4:11592]   File "/var/www/html/flaskapp/flaskapp.py", line 37, in predict
[Thu Nov 02 22:15:21.936924 2017] [wsgi:error] [pid 11753:tid 140263731681024] [client 130.245.192.4:11592]     import numpy as np
[Thu Nov 02 22:15:21.936931 2017] [wsgi:error] [pid 11753:tid 140263731681024] [client 130.245.192.4:11592] ImportError: No module named numpy

score 1 · Accepted Answer · answered Nov 02 '17 at 22:42

The issue was the apache2 was not able to access the directory. Sadly, there's a very little comment in the question posted above, and the accepted answer isn't the right one.

Adding the solution here if it helps someone:

sudo chown -R www-data:www-data /home

score 0 · Answer 2 · answered Jun 26 '20 at 12:48

0

sudo groupadd www-data
sudo usermod -a -G www-data ubuntu
sudo chown root:www-data /home
sudo chmod -R 775 /home

This should be able to run

answered Jun 26 '20 at 12:48

Niza Siwale

2,390
1
18
20

Can't import modules on EC2 - Flask App

2 Answers2