122

What sort of directory structure should one follow when using virtualenv? For instance, if I were building a WSGI application and created a virtualenv called foobar I would start with a directory structure like:

/foobar
  /bin
    {activate, activate.py, easy_install, python}
  /include
    {python2.6/...}
  /lib
    {python2.6/...}

Once this environment is created, where would one place their own:

  • python files?
  • static files (images/etc)?
  • "custom" packages, such as those available online but not found in the cheese-shop?

in relation to the virtualenv directories?

(Assume I already know where the virtualenv directories themselves should go.)

nbro
  • 15,395
  • 32
  • 113
  • 196
Phillip B Oldham
  • 18,807
  • 20
  • 94
  • 134
  • 8
    @jkp: I disagree. How you layout a python application is a different matter from how you locate that application within a virtualenv for development purposes. It's related, but not the same. Please do not close as duplicate. – jcdyer Nov 23 '09 at 16:09

4 Answers4

99

virtualenv provides a python interpreter instance, not an application instance. You wouldn't normally create your application files within the directories containing a system's default Python, likewise there's no requirement to locate your application within a virtualenv directory.

For example, you might have a project where you have multiple applications using the same virtualenv. Or, you may be testing an application with a virtualenv that will later be deployed with a system Python. Or, you may be packaging up a standalone app where it might make sense to have the virtualenv directory located somewhere within the app directory itself.

So, in general, I don't think there is one right answer to the question. And, a good thing about virtualenv is that it supports many different use cases: there doesn't need to be one right way.

nbro
  • 15,395
  • 32
  • 113
  • 196
Ned Deily
  • 83,389
  • 16
  • 128
  • 151
  • 11
    Agreed. I use virtualenv for everything I do, and I never place files inside the virtualenv directory. Virtualenv need have no impact on your project structure; just activate the virtualenv (or use its bin/python), and work on your files wherever you'd have them anyway. – Carl Meyer Nov 23 '09 at 20:31
  • I also agree wholeheartedly. The only time I *ever* touch any files inside my virtualenv (I use `virtualenvwrapper`) is when I want to edit the `postactivate` and `postdeactivate` hooks. – Thane Brimhall Oct 04 '13 at 16:35
  • 2
    Question would be better served with concrete, practical examples of different options including trade offs as seen in other answers in this question. – andyfeller Mar 08 '18 at 15:26
  • 3
    It is cleaner to keep your project separate to the `virtualenv` directory, but comparing `virtualenv` to system python is unhelpful, because the purpose of `virtualenv` is to fix broken dependencies and isolate projects so they can use different package versions and even python versions (I realise this was written pre-python3). Allowing apps to share a `virtualenv` is using `virtualenv` as though it were system python, leaving apps vulnerable to the same issues virtualenv is designed to solve. `There should be one obvious way to do it`; logically that should be 1:1 – Davos May 08 '18 at 04:38
  • @Ned: Trying to acquire some best practices, but it is still unclear: if you have dozens of projects, each with its own virtualenv then how do you keep track of which project is used with which virtualenv? Add tiny shell scripts in the root of each folder with the name of the virtualenv you use it with? – ccpizza Jul 18 '18 at 10:40
64

If you only have a few projects every so often, nothing stops you from creating a new virtualenv for each one, and putting your packages right inside:

/foobar
  /bin
    {activate, activate.py, easy_install, python}
  /include
    {python2.6/...}
  /lib
    {python2.6/...}
  /mypackage1
    __init__.py
  /mypackage2
    __init__.py

The advantage of this approach is that you can always be sure to find find the activate script that belongs to the project inside.

$ cd /foobar
$ source bin/activate
$ python 
>>> import mypackage1
>>>

If you decide to be a bit more organized, you should consider putting all your virtualenvs into one folder, and name each of them after the project you are working on.

  /virtualenvs
    /foobar
      /bin
        {activate, activate.py, easy_install, python}
      /include
        {python2.6/...}
      /lib
        {python2.6/...}
  /foobar
    /mypackage1
      __init__.py
    /mypackage2
      __init__.py

This way you can always start over with a new virtualenv when things go wrong, and your project files stay safe.

Another advantage is that several of your projects can use the same virtualenv, so you don't have to do the same installation over and over if you have a lot of dependencies.

$ cd /foobar
$ source ../virtualenvs/foobar/bin/activate
$ python 
>>> import mypackage2
>>>

For users that regularly have to set up and tear down virtualenvs it would make sense to look at virtualenvwrapper.

http://pypi.python.org/pypi/virtualenvwrapper

With virtualenvwrapper you can

* create and delete virtual environments

* organize virtual environments in a central place

* easily switch between environments

You no more have to worry about where your virtualenvs are when working on the projects "foo" and "bar":

  /foo
    /mypackage1
      __init__.py
  /bar
    /mypackage2
      __init__.py

This is how you start working on project "foo":

$ cd foo
$ workon
bar
foo
$ workon foo
(foo)$ python
>>> import mypackage1
>>>

Then switching to project "bar" is as simple as this:

$ cd ../bar
$ workon bar
(bar)$ python
>>> import mypackage2
>>>

Pretty neat, isn't it?

Maik Röder
  • 986
  • 7
  • 6
  • I *strongly* agree with this answer about using `virtualenvwrapper`. It neatly abstracts the virtualenv away while still giving you all the benefits. – Thane Brimhall Oct 04 '13 at 16:31
  • 5
    But strongly *disagree* about EVER putting your code inside the virtual environment. If you want it "near" the project on the filesystem then put a `venv/` directory on the same level as the project's `BASE_DIR`. – Rob Grant Aug 15 '14 at 11:09
32

Because virtualenvs are not relocatable, in my opinion it is bad practice to place your project files inside a virtualenv directory. The virtualenv itself is a generated development/deployment artifact (sort of like a .pyc file), not part of the project; it should be easy to blow it away and recreate it anytime, or create a new one on a new deploy host, etc.

Many people in fact use virtualenvwrapper, which removes the actual virtualenvs from your awareness almost completely, placing them all side-by-side in $HOME/.virtualenvs by default.

Praveen
  • 6,872
  • 3
  • 43
  • 62
Carl Meyer
  • 122,012
  • 20
  • 106
  • 116
  • 1
    Completely agree it is bad practice, great to point out that it should be easy to blow away and recreate, especially for testing deployments and whittling away unneeding requirements packages. Just want to add that virtualenv is possible to relocate using e.g. `virtualenv --relocatable myvenv` see https://stackoverflow.com/a/6628642/1335793 just because you can doesn't mean you should though. – Davos May 08 '18 at 04:19
3

If you give your project a setup.py, pip can import it from version control directly.

Do something like this:

$ virtualenv --no-site-packages myproject
$ . myproject/bin/activate
$ easy_install pip
$ pip install -e hg+http://bitbucket.org/owner/myproject#egg=proj

The -e will put the project in myproject/src, but link it to myproject/lib/pythonX.X/site-packages/, so any changes you make will get picked up immediately in modules that import it from your local site-packages. The #egg bit tells pip what name you want to give to the egg package it creates for you.

If you don't use --no-site-packages, be careful to specify that you want pip to install into the virtualenv with the -E option

jcdyer
  • 18,616
  • 5
  • 42
  • 49