156

I want to "activate" a virtualenv in a systemd service file.

I would like to avoid having a shell process between the systemd process and the python interpreter.

My current solution looks like this:

[Unit]
Description=fooservice
After=syslog.target network.target

[Service]
Type=simple
User=fooservice
WorkingDirectory={{ venv_home }}
ExecStart={{ venv_home }}/fooservice --serve-in-foreground
Restart=on-abort
EnvironmentFile=/etc/sysconfig/fooservice.env

[Install]
WantedBy=multi-user.target

/etc/sysconfig/fooservice.env

PATH={{ venv_home }}/bin:/usr/local/bin:/usr/bin:/bin
PYTHONIOENCODING=utf-8
PYTHONPATH={{ venv_home }}/...
VIRTUAL_ENV={{ venv_home }}

But I am having trouble. I get ImportErrors since some entries in sys.path are missing.

Josh Correia
  • 3,807
  • 3
  • 33
  • 50
guettli
  • 25,042
  • 81
  • 346
  • 663

5 Answers5

186

The virtualenv is "baked into the Python interpreter in the virtualenv". This means you can launch python or console_scripts directly in that virtualenv and don't need to activate the virtualenv first or manage PATH yourself.:

ExecStart={{ venv_home }}/bin/fooservice --serve-in-foreground

or

ExecStart={{ venv_home }}/bin/python {{ venv_home }}/fooservice.py --serve-in-foreground

and remove the EnvironmentFile entry.

To verify that it is indeed correct you can check sys.path by running

{{ venv_home }}/bin/python -m site

and comparing the output to

python -m site
gilbertbw
  • 634
  • 2
  • 9
  • 27
Nils Werner
  • 34,832
  • 7
  • 76
  • 98
  • Yes, sys.path is correct. Thank you. I checked the difference between os.environ. It seems that VIRTUAL_ENV is not set, if I use `{{ venv_home }}/bin/python`. Can this make trouble? – guettli May 30 '16 at 12:49
  • I don't know for certain but I have never had issues with `VIRTUAL_ENV` not being set. – Nils Werner May 31 '16 at 10:08
  • 5
    good point Nils. Btw, fooservice.py doesn't make sense to be inside venv_home directory, I suppose its a typo in the question. – stelios Jan 29 '17 at 18:06
  • 5
    Note the suggested print commands are not compatible with Python 3. If you are using at least python 2.4, you can alternatively just use: `python -m site` to get a nicely formatted output of the sys.path variable along with additional information. – Mark Edington Mar 06 '17 at 21:40
  • 3
    Neat, I didn't know about `python -m site`. I have adjusted my answer. – Nils Werner Mar 07 '17 at 08:56
  • Worked like a charm for me. Thanks. – Shubham Namdeo Oct 27 '17 at 01:52
  • 2
    This approach doesn't work when the python process you're starting expects the virtualenv to be activated. For example, consider `{{ venv_home }}/bin/python -m newrelic.admin run-program python -m myapp`. That will launch python in the venv, invoke New Relic, and then it will fail to launch the app because `python` is not in the path because the venv is not activated. – Jason R. Coombs Jan 12 '18 at 18:51
  • Have you tried `{{ venv_home }}/bin/python -m newrelic.admin run-program {{ venv_home }}/bin/python -m myapp`? – Nils Werner Jan 12 '18 at 19:02
  • Also appears to fail with python3 when the (`python3 -mvenv` created) venv has third party modules not in the python3 stdlib? See here: https://gist.github.com/brainstorm/bff8b439072ffadbd95ae901c659a6b5 – brainstorm Feb 27 '18 at 07:20
  • @brainstorm I cannot reproduce your issues on OSX: https://gist.github.com/nils-werner/d98aff2b865dac9375570340ce635844. Maybe it has to with pcgr – Nils Werner Feb 27 '18 at 08:44
  • 1
    @NilsWerner I ended up solving it by spawning a shell, nothing else worked on Ubuntu 17.10: https://github.com/umccr/pcgr-deploy/blob/master/ansible/files/pcgr.service.j2#L9 ... please ignore the jinja2 templating for ansible, it expands correctly when deployed. – brainstorm Feb 28 '18 at 00:53
  • 21
    For those wondering if this is ninja2....no, the double curly-braces are just placeholders the OP invented: https://superuser.com/questions/1209919/what-do-double-curly-braces-mean-in-systemd-scripts – ankostis Jul 06 '18 at 21:03
  • what exactly does --serve-in-foreground do? – Shankara Narayana Dec 22 '20 at 16:23
  • That's a parameter of the Python server the OP invented. – Nils Werner Dec 22 '20 at 16:42
  • When I use the `{{ venv_home }}` syntax, I get errors that it's not a path. – FilBot3 Feb 23 '21 at 22:12
33

While the path for libraries is indeed baked into the python interpreter of the virtualenv, I've had issues with python tools that were using binaries installed in that virtualenv. For instance, my apache airflow service wouldn't work because it couldn't find the gunicorn binary. To work around this, here's my ExecStart instruction, with an Environment instruction (which sets an environment variable for the service alone).

ExecStart={{ virtualenv }}/bin/python {{ virtualenv }}/bin/airflow webserver
Environment="PATH={{ virtualenv }}/bin:{{ ansible_env.PATH }}"

ExecStartexplicitly uses the python interpreter of the virtualenv. I'm also adding a PATH variable, which adds the binary folder of the virtualenv before the system PATH. That way, I get the desired python libraries as well as binaries.

Note that I'm using ansible to build this service, ergo the curly braces of jinja2.

Alexis Lessard
  • 553
  • 6
  • 7
  • 2
    Literally came looking for how to set the environment in a service because airflow couldn't find gunicorn. Was NOT disappointed! Thank you! I'm up and running. – pyFiddler Jun 17 '21 at 19:18
  • Exactly , I was in need of Airflow automation through Ansible . Thank you so much :) – Manju N Jul 07 '21 at 06:42
6

In my case I just tried to add environment variables required for Flask, for instance

[Service]
Environment="PATH=/xx/yy/zz/venv/bin"
Environment="FLASK_ENV=development"
Environment="APP_SETTINGS=config.DevelopmentConfig"

I was using virtualenv so /xx/yy/zz/venv/bin is the path of virtualenv folder.

Sebastian Cardona Osorio
  • 475
  • 7
  • 8
3

I'm not using virtualenv but pyenv: here is it just to use the real .pyenv path in the shebang and make sure it is in the PATH

Ex: pyenv activate flask-prod for user mortenb which is running in prod 

/home/mortenb/.pyenv/versions/flask-prod/bin/python --version
Python 3.6.2

Then to my flask scripts starting in systemd *.service I add the following shebang:

#!/home/mortenb/.pyenv/versions/flask-prod/bin/python3
MortenB
  • 2,749
  • 1
  • 31
  • 35
0
ExecStart=cd /root/app/working-directory && poetry run python my_app.py
马卓群
  • 1
  • 1
  • 1
    As it’s currently written, your answer is unclear. Please [edit] to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers [in the help center](/help/how-to-answer). – Community May 30 '23 at 05:40