9

I'm trying to deploy django with uwsgi, and I think I lack understanding of how it all works. I have uwsgi running in emperor mode, and I'm trying to get the vassals to run in their own virtualenvs, with a different python version.

The emperor configuration:

[uwsgi]
socket = /run/uwsgi/uwsgi.socket
pidfile = /run/uwsgi/uwsgi.pid
emperor = /etc/uwsgi.d
emperor-tyrant = true
master = true
autoload = true
log-date = true
logto = /var/log/uwsgi/uwsgi-emperor.log

And the vassal:

uid=django
gid=django
virtualenv=/home/django/sites/mysite/venv/bin
chdir=/home/django/sites/mysite/site
module=mysite.uwsgi:application
socket=/tmp/uwsgi_mysite.sock
master=True

I'm seeing the following error in the emperor log:

Traceback (most recent call last):
  File "./mysite/uwsgi.py", line 11, in <module>
    import site
ImportError: No module named site

The virtualenv for my site is created as a python 3.4 pyvenv. The uwsgi is the system uwsgi (python2.6). I was under the impression that the emperor could be any python version, as the vassal would be launched with its own python and environment, launched by the master process. I now think this is wrong.

What I'd like to be doing is running the uwsgi master process with the system python, but the various vassals (applications) with their own python and their own libraries. Is this possible? Or am I going to have to run multiple emperors if I want to run multiple pythons? Kinda defeats the purpose of having virtual environments.

Josh Smeaton
  • 47,939
  • 24
  • 129
  • 164
  • In the end did you stack the emperors or you build different plugin? – Kim Stacks Apr 09 '15 at 08:38
  • @KimStacks I didn't do either. I built uwsgi with the version of python my application was going to support, and didn't run different python uwsgis on the same system. If I had to do it though, I'd build the different versions I think. – Josh Smeaton Apr 09 '15 at 13:30
  • I need to run different django apps that require different python environments. What would you suggest? – Kim Stacks Apr 13 '15 at 01:58
  • @KimStacks I'd build different plugins as roberto has done in his answer below. This is OK if you can automate it, but not good if you have to do it manually over and over again. You might want to look at using gunicorn instead, which is easier to deploy but not as performant for high traffic as far as I know. – Josh Smeaton Apr 13 '15 at 04:02
  • 1
    For the record, my recommendation if you need to support multiple versions of python without complicated building is to forget about using emperor, and instead write a service (systemd/upstart, whatever) that executes the uwsgi installed into the virtualenv. – Josh Smeaton Oct 21 '15 at 03:20

2 Answers2

8

The "elegant" way is building the uWSGI python support as a plugin, and having a plugin for each python version:

(from uWSGI sources)

make PROFILE=nolang

(will build a uWSGI binary without language support)

PYTHON=python2.7 ./uwsgi --build-plugin "plugins/python python27"

will build the python27_plugin.so that you can load in vassals

PYTHON=python3 ./uwsgi --build-plugin "plugins/python python3"

will build the plugin for python3 and so on.

There are various way to build uWSGI plugins, the one i am reporting is the safest one (it ensure the #ifdef are honoured).

Having said that, having a uWSGI Emperor for each python version is viable too. Remember Emperor are stackable, so you can have a generic emperor spawning one emperor (as its vassal) for each python version.

Veedrac
  • 58,273
  • 15
  • 112
  • 169
roberto
  • 12,723
  • 44
  • 30
  • You are absolutely right. I asked in the #uwsgi irc and was given this information too. I was unaware that uwsgi was built in this way. I had simply `pip install uwsgi`. I'm looking to automate the deployment of uwsgi, and I hate building my own software :P – Josh Smeaton May 26 '14 at 23:54
  • The idea of stacking emperor's is brilliant though, I'll look into that more closely. – Josh Smeaton May 26 '14 at 23:54
  • 1
    Quick follow up - but how would you tell the first emperor to spawn the second emperor? How do you provide the "uwsgi" path to execute? – Josh Smeaton May 27 '14 at 01:06
  • 1
    http://uwsgi-docs.readthedocs.org/en/latest/Emperor.html#running-non-uwsgi-apps-or-using-alternative-uwsgis-as-vassals – roberto May 27 '14 at 04:07
  • @roberto could you elaborate on how to have a uWSGI Emperor spawning one emperor for each python version? – Kim Stacks Aug 25 '15 at 05:55
2

Pip install uWSGI

One option would be to simply install uWSGI with pip in your virtualenvs and start your services separately:

pip install uwsgi
~/.virtualenvs/venv-name/lib/pythonX.X/site-packages/uwsgi --ini path/to/ini-file

Install uWSGI from source and build python plugins

If you want a system-wide uWSGI build, you can build it from source and install plugins for multiple python versions. You'll need root privileges for this.

First you may want to install multiple system-wide python versions.

Make sure you have any dependencies installed. For pcre, on a Debian-based distribution use:

apt install libpcre3 libpcre3-dev

Download and build the latest uWSGI source into /usr/local/src, replacing X.X.X.X below with the package version (e.g. 2.0.19.1):

wget http://projects.unbit.it/downloads/uwsgi-latest.tar.gz
tar vzxf uwsgi-latest.tar.gz
cd uwsgi-X.X.X.X/
make PROFILE=nolang

Symlink the versioned folder uwsgi-X.X.X.X to give it the generic name, uwsgi:

ln -s /usr/local/src/uwsgi-X.X.X.X /usr/local/src/uwsgi

Create a symlink to the build so it's on your PATH:

ln -s /usr/local/src/uwsgi/uwsgi /usr/local/bin

Build python plugins for the versions you need:

PYTHON=pythonX.X ./uwsgi --build-plugin "plugins/python pythonXX"

For example, for python3.8:

PYTHON=python3.8 ./uwsgi --build-plugin "plugins/python python38"

Create a plugin directory in an appropriate location:

mkdir -p /usr/local/lib/uwsgi/plugins/

Symlink the created plugins to this directory. For example, for python3.8:

ln -s /usr/local/src/uwsgi/python38_plugin.so /usr/local/lib/uwsgi/plugins

Then in your uWSGI configuration (project.ini) files, specify the plugin directory and the plugin:

plugin-dir = /usr/local/lib/uwsgi/plugins
plugin = python38

Make sure to create your virtualenvs with the same python version that you created the plugin with. For example if you created python38_plugin.so with python3.8 and you have plugin = python38 in your project.ini file, then an easy way to create a virtualenv with python3.8 is with:

python3.8 -m virtualenv path/to/project/virtualenv
blue-cat
  • 111
  • 1
  • 3