Django / Python error with Passenger on Dreamhost

Question

Hi Stackoverflow community,

I have recently been experimenting with setting up Django on dreamhost (using Passenger) and have come across a problem I can't seem to fix.

First of all, I have used the following guides to learn and set everything up:

1) Update new Django and Python 2.7.* with virtualenv on Dreamhost (with passenger)

2) http://www.ricksresources.com/2012/07/writing-wsgi-apps-on-dreamhost-shared-hosting/

3) http://wiki.dreamhost.com/Django

And here is a summary of the steps I have completed:

1. Setup a subdomain sub.domain.com on dreamhost.com
2. Enabled Passenger for the subdomain in order to take advantage of WSGI
3. Installed Python 3.4 into my $HOME/Python3.4 directory
4. Installed ez_setup.py in order to get pip
5. Installed virtualenv in order to create a separate environment for the django project
6. Created a virtualenv in $HOME/sub.domain.com/env/
7. Downloaded and installed Django 1.7 using pip
8. Started a djangoproject called djtest

9. Created a passenger_wsgi.py file in the subdomain root directory: sub.domain.com with the following code:

   
   import sys, os
   cwd = os.getcwd()
   sys.path.append(cwd)
   
   project_location = cwd + "/djtest"
   
   sys.path.insert(0,project_location)
   
   PYTHON_EXE = '/home/USERNAME/DOMAIN/env/bin/python3'
   
   if sys.executable != PYTHON_EXE:
       os.execl(PYTHON_EXE, PYTHON_EXE, *sys.argv)
   
   sys.path.insert(0,'home/USERNAME/DOMAIN/env/bin')
   sys.path.insert(0,'home/USERNAME/DOMAIN/env/lib/python3.4/site-packages/django')
   sys.path.insert(0,'home/USERNAME/DOMAIN/env/lib/python3.4/site-packages')
   
   os.environ['DJANGO_SETTINGS_MODULE'] = "djtest.settings"
   
   from django.core.wsgi import get_wsgi_application
   application = get_wsgi_application()
   

After I try to access the domain I get the following error:

python3.4: error while loading shared libraries: libpython3.4m.so.1.0: cannot open shared object file: No such file or directory

Please note however, that Passenger + Python do work together since putting the following code into passenger_wsgi.py renders it correctly on the web:

def application(environ, start_response):
    start_response('200 OK', [('content-type', 'text/html')])
    return ['Hello world!']

I came across a few articles which are stating it is a problem related to not correctly specifying the public library path and the python path, i.e.: LD_LIBRARY_PATH and PYTHONPATH for example: Python: ImportError: No module named os

So I am guessing my mistake is somewhere around that and how to correctly specify those when running a setup in virtualenv.

Any help would be strong appreciated :)

Thank you!

~Aivoric

---

EDIT

It seems I found a solution...well django now fires up correctly...but I still don't know why the environment variables are not working. Why doesn't the LD_LIBRARY_PATH set by itself without having to hard-code it into the code?

So what I did was insert the line:

os.environ['LD_LIBRARY_PATH'] = '/home/USERNAME/Python34/lib'

Into this code:

import sys, os
cwd = os.getcwd()
sys.path.append(cwd)

project_location = cwd + "/djtest"

sys.path.insert(0,project_location)
os.environ['LD_LIBRARY_PATH'] = '/home/USERNAME/Python34/lib'

#Switch to new python
PYTHON_EXE = '/home/USERNAME/DOMAIN/env/bin/python3'

if sys.executable != PYTHON_EXE:
    os.execl(PYTHON_EXE, PYTHON_EXE, *sys.argv)

sys.path.insert(0,'home/USERNAME/DOMAIN/env/bin')
sys.path.insert(0,'home/USERNAME/DOMAIN/env/lib/python3.4/site-packages/django')
sys.path.insert(0,'home/USERNAME/DOMAIN/env/lib/python3.4/site-packages')

os.environ['DJANGO_SETTINGS_MODULE'] = "djtest.settings"

from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()

I think the error was being generated when os.execl was being used but it was unable to find the shared libraries: libpython3.4m.so.1.0

If someone can tell me why that might have happened it will make me very happy :) thank you!

Answer 1

I see you are messing up python interpreters along the installation.
Try to do the following:

Run your django project in local. I mean install everything you need using a virtualenv, make sure your project works on local.

Then, still in your local run a pip freeze > requirements.txt

Move to your Dreamhost hosting, let's assume you have:

/path/to/project/           # Root for tmp/restart and passenger_wsgi.py 
/path/to/project/public     # This is the root of your django project
/path/to/project/public/env # Your virtual env

Upload your requirements.txt file

Run:

cd /path/to/project/public

rm -rf env
virtualenv env
source env/bin/activate

which python # make sure your interpreter is in the virtualenv 
pip install -r requirements.txt

Passenger

Regarding passenger_wsgi.py, a minimal example that works under the detailed installation:

import myproject.wsgi
application = myproject.wsgi.application

If the previous example doesn't work, it means some python handlers are not properly installed.
It is important to do all the libraries installation with the python interpreter of the virtual environment.

Just in case, here it is the example I use.

Note I had to add a egg-info folder which was not being properly located by other module __imports__

import os
import sys

cwd = os.getcwd()
env_dir = os.path.join(cwd,'public',  'venv')
project_dir = os.path.join(cwd, 'public')    

# Use the python executable from inside our virtualenv
# https://help.dreamhost.com/hc/en-us/articles/215769548
INTERP = os.path.join(env_dir, 'bin', 'python')
if sys.executable != INTERP:
    os.execl(INTERP, INTERP, *sys.argv)

# Add virtualenv packages to the start of the path
sys.path.insert(0, os.path.join(env_dir, 'bin'))
sys.path.insert(0, os.path.join(env_dir, 'lib', 'python2.7', 'site-packages'))
sys.path.insert(0, os.path.join(env_dir, 'lib', 'python2.7', 'site-packages','zope.interface-3.6.0-py2.7.egg-info'))
sys.path.insert(0, os.path.join(env_dir, 'lib', 'python2.7', 'site-packages', 'django'))

# Add brickwatch django project to the *end* of the path
# (so it will be checked last).
sys.path.append(project_dir)

# Set environment variables for django to use
os.environ['DJANGO_SETTINGS_MODULE'] = 'myproject.settings'

from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()

Restart

In order to restart the passenger profile, include an empty file in /path/to/project/tmp called restart.txt.

Each time you modify passenger_wsgi.py, run a:
touch /path/to/project/tmp/restart.txt

It will tell passenger to reload the changes.

It is recommended to pkill python each time the application is modified to avoid problems with python and loaded files.

Application Logging

Have a look at more versions of passenger_wsgi.py file here:
https://help.dreamhost.com/hc/en-us/articles/215769548-Passenger-and-Python-WSGI