Separating development and production dependencies with virtualenv

Question

I have a simple flask app, where I use virtualenv for managing my dependencies. However, I would like to separate my dependencies. For testing I use Flask-Testing, but this is a dependency I wouldn't want to include in my production environment.

How can separate my dependencies, so that I have separate dependencies for development and for production?

You can create different files for each environment. Check [this answer](https://stackoverflow.com/a/20720019/5741172) — Sohaib Farooqi, Nov 15 '17 at 15:58

score 7 · Accepted Answer · answered Nov 15 '17 at 15:59

7

You can have multiple requirements.txt files for different environments. eg. requirements_dev.txt, requirements_prod.txt...

These files would have different pinned (having versions as well) packages. They can be generated by manually pip installing packages and then piping the output of pip freeze to a file.

answered Nov 15 '17 at 15:59

tayfun

3,065
1
19
23

If I do that how can I use the different environments on my local machine? Can I switch environments based on my different requirement.txt files? I will have installed every single package which are in any of my requirement.txt file and I won't know for example if my testing environment lacks a package only if I run the tests on the server. – handris Nov 15 '17 at 16:05
@handris Generally requirements-*.txt files will only have extra packages that you don't want elsewhere. For example requirements-dev.txt will have debuggers etc. whereas requirements_test.txt will also contain test libraries. So it should not be a problem if you install them all (so you will always install base/main file, but others depending on test/dev/prod environment). Otherwise you can also install different requirements files in different virtualenv, using `pip install -r requirement_file.txt`. – tayfun Nov 15 '17 at 16:10

score 3 · Answer 2 · answered Jan 26 '21 at 19:57

pip-tools is the right tool. I had the same problem so I solved it in this easy way.

From the site:

If you have different environments that you need to install different but compatible packages for, then you can create layered requirements files and use one layer to constrain the other.

For example, if you have a Django project where you want the newest 2.1 release in production and when developing you want to use the Django debug toolbar, then you can create two *.in files, one for each layer:

# requirements.in
django<2.2

At the top of the development requirements dev-requirements.in you use -c requirements.txt to constrain the dev requirements to packages already selected for production in requirements.txt.

# dev-requirements.in
-c requirements.txt
django-debug-toolbar

First, compile requirements.txt as usual:

$ pip-compile
#
# This file is autogenerated by pip-compile
# To update, run:
#
#    pip-compile
#
django==2.1.15
    # via -r requirements.in
pytz==2019.3
    # via django

Now compile the dev requirements and the requirements.txt file is used as a constraint:

$ pip-compile dev-requirements.in
#
# This file is autogenerated by pip-compile
# To update, run:
#
#    pip-compile dev-requirements.in
#
django-debug-toolbar==2.2
    # via -r dev-requirements.in
django==2.1.15
    # via
    #   -c requirements.txt
    #   django-debug-toolbar
pytz==2019.3
    # via
    #   -c requirements.txt
    #   django
sqlparse==0.3.0
    # via django-debug-toolbar

As you can see above, even though a 2.2 release of Django is available, the dev requirements only include a 2.1 version of Django because they were constrained. Now both compiled requirements files can be installed safely in the dev environment.

To install requirements in production stage use:

$ pip-sync

You can install requirements in development stage by:

$ pip-sync requirements.txt dev-requirements.txt

Separating development and production dependencies with virtualenv

2 Answers2