23

I have a dilemma setting up a local development projet structure. Here is my setup:

  • Python 2.7
  • Django 1.9
  • Mac OSX El Capitan 10.11
  • MySQL 5.7

I made a "Mistake" of setting my project globally instead of in a virtual environment (using 'pip' to install everything in />). After reading this article I still don't get all the steps. Is this correct:

  1. I install global python ( pip, virtualenv in '/>' )
  2. I then go to a location where my projects will reside, like /users/user/documents/projects/project1 and from within 'project1' I use 'virtualenv' to create a virtual environment for this project (this creates a /virtual env/ folder inside /project1/ folder)
  3. activate this virtual environment and pip install django
  4. then from within newly created /virtual env/ folder I startprojectwhich creates another /project1/ folder within /virtual env/ folder
  5. with virtual environment still activated in the current shell session, I proceed with creating my scripts, site and app files

Ad 2. should the virtualenv folder be INSIDE the main "project1" folder or should it encompass it?

Ad 4. Is this correct or can I do it without activating virtual environment first?

My structure currently looks like this (starts from the root: /users/myUser/documents/projects/):

/project1/
    /website1/
        /static/
        /templates/
        __init.py__
        settings.py
        urls.py
        views.py
        wsgi.py
Avión
  • 7,963
  • 11
  • 64
  • 105
Alexander Starbuck
  • 1,139
  • 4
  • 18
  • 31

2 Answers2

24

Common solution is to keep virtual environments and projects in separate folders, e.g. have /users/myUser/.venvs for virtual environments and /users/myUser/documents/projects/ for projects. In other aspects you got it pretty much right yourself. So:

  1. You need to install global Python and virtualenv.
  2. Create directoriy for virtual environments, e.g. run mkdir /users/myUser/.venvs.
  3. Create the virtual environment for your project, virtualenv /users/myUser/.venvs/project1_venv.
  4. Activate the environment for your current shell session /users/myUser/.venvs/project1_venv/bin/activate.
  5. Install django and anything else in this environment pip install django, or better use requirements.txt file to keep track of all project dependencies.
  6. Deactivate the environment, run deactivate.

Now when you'll want to run your project using created virtual environment, in your console window run /users/myUser/.venvs/project1_venv/bin/activate and then python /users/myUser/documents/projects/project1/manage.py runserver. You can activate the venv from any directory, it's activated for current shell window and any python ... run in that window after activation will use this virtual environment. The activation script modifies environment variables in a way, so that the interpreter and libraries from venv are used instead of global ones. (Though there are options to use global ones too.)

Nikita
  • 6,101
  • 2
  • 26
  • 44
  • I would like to accept this reply as it is more elaborate and newb-friendly but the one above was faster and it made me "connect the dots" on my own. I dont know what is the SO etiquette regarding this? – Alexander Starbuck Mar 11 '16 at 11:15
  • 3
    @AlexStarbuck, You'll have to research the [StackOverflow Meta](http://meta.stackoverflow.com/) to find the answer on your own. But in general, all the answers stay here, and the idea behind marking one answer - the one that fits your question best, is that other people with similar problem could find the solution faster. So in general it's not about the 'fastest' answer, but about the most 'helpful' one. Anyway, nice that I was able to help. – Nikita Mar 11 '16 at 11:20
  • Somehow when running the server from the venv activated for my project is not calling libraries from the venv that I do have but those from the project, and this is already wrong as should not be the case. The lib in the folder of my project are missing, because my project is been a clone of a repo. I am not sure but I don't think is due the fact that I am running this from ubuntu and the project initially is been developed in Windows, but I checked this and there is nothing to care adjust about this https://stackoverflow.com/questions/12033861/cross-platform-interface-for-virtualenv – Carmine Tambascia May 24 '18 at 18:51
  • @CarmineTambascia, I'm sorry, I don't quite get your issue. But if it's about replicating the venv from the repo, that you downloaded, then the link you provided has a pretty good answer (see comments also). venvs are replicatable, you don't need to store it in the repo, or download. Once you download the repo, make a new venv yourself (it will be empty, without the libs a project needs). Then activate the venv the way it is done on your OS and install the required packages there by running `pip install -r requirements.txt` in the activated venv. – Nikita May 24 '18 at 19:16
  • If there're no `requirements.txt` in that repo (which is usually not the case in a good python project), then you have to find out all the libs, that the project requires and create a requirements file yourself, or just install those libs one by one by running `pip install lib_name` in the activated venv. Also, some libs might require compilation, so those can't just be easily moved from one system to another, but could be installed (and compiled as part of installation step) on the required system. – Nikita May 24 '18 at 19:21
  • @Nikita sorry I was not that clear. I indeed create a new venv, then installed the dependencies from requirements.txt and also all went smoth. The problem is when I run python /users/myUser/documents/projects/project1/manage.py runserver I got /.virtualenv/env/bin/python: can't find '__main__' module in '/home/carmine/Project/repo'. So it is looking at the project lib folder(more specifically at site-packages) that does not exist. This should not be the case, it should look for the venv lib. Maybe I am missing something – Carmine Tambascia May 24 '18 at 21:19
  • So basically is not behaving such "The activation script modifies environment variables in a way, so that the interpreter and libraries from venv are used instead of global ones." – Carmine Tambascia May 25 '18 at 08:24
  • I have found out the reason of the issue, though I run inside the venv activated the pip install project_path/requirements.txt it seems it installed everything in the base machine environment and not in the venv. – Carmine Tambascia May 25 '18 at 08:46
  • So I did manage to install requirements into the venv as I had to use venv/pip. Of course now I still the error as in my previous project I put git inside Scripts/src and so it cannot see lib folder.I found this very inconvenient as once activate it should use modules and libraries from venv but does not seems so. – Carmine Tambascia May 25 '18 at 09:46
  • @CarmineTambascia, Take a note, that the venv is activated for current process only, i.e. the shell process where you run the activation script. So if you active the venv in one terminal window and run `pip install` in another window, then pip won't see the activated venv, it'll see the systems python (and site-packages). Same is true for `manage.py runserver`. So you have to run all commands in the same terminal window, but of course you can activate venv in several terminal windows at the same time and all of them will "see" the same activated venv. – Nikita May 25 '18 at 11:20
  • @Nikita, yes know this, I activated and run both pip3(I put Python 3.5 in the venv) and manage.py runserver in the same shell process. As wrote above I had to write the venv pip path in order to install requirements into the venv. In Windows I did not had this problem though. How I can check venv is "created" as it should except that already verified that when is activated which python return the venv path and "python" return indeed 3.5. When deactivate it return the 2.7 that is the base on the system. This is why it seems to me everything should be fine. – Carmine Tambascia May 25 '18 at 14:51
  • @CarmineTambascia I remember, I've encountered issues, when python was from venv, but pip used system python, though it was quite sometime ago. Anyway, on Linux you can check which pip (and python) is used by running `which pip`. On Windows I believe there should be an analog in the form of `where.exe pip` or `where pip`. Alternatively you can check the order of directories in the `PATH` environment variable, because prepending venv directory there is basically what happens, when a venv is activated. – Nikita May 28 '18 at 05:07
  • @Nikita. Thanks for your answer. I found out that unfortunately my original venv is corrupt. Indeed I am able both in Windows, creating a new venv to run the project folder from within the new working venv as was previously, at least python manage.py runserver put up the dev server and also I can see what I already "created". Thank you for the support though – Carmine Tambascia May 28 '18 at 07:33
18

It doesn't really matter where you store your virtual environment. Find a project structure that works for you.

I wouldn't put the virtual env inside the project, because you shouldn't check it into version control (although you could use an ignore). Normally, you just need to check in your requirements file, so that you can recreate the environment.

I wouldn't put the project inside the virtual env, because virtual envs are disposable. You might want to destroy the virtual env without destroying the project. Also, you might want to run the same project under different virtual envs e.g. test your code on Django 1.8 and 1.9 before upgrading.

You might find virtualenvwrapper useful. It has some tools that make it easy to create and switch between virtual environments. It stores all of your virtual environments in one place, so you don't have to worry about where to put them.

Is this correct or can I do it without activating virtual environment first?

You should activate the virtual environment and install django before you create / work on your project.

Alasdair
  • 298,606
  • 55
  • 578
  • 516
  • Thank you Alasdair! So if I read you correctly: project folder structure and virtual env folder structure should basically be two completely separate things? I can 1. create virtualenv on `C:`, 2. activate it, 3. change directory to `D:`, 4. with virtualenv activated, start creating my project folder structure on `D:` and keep working on it? – Alexander Starbuck Mar 11 '16 at 09:41
  • 1
    Yes. The key thing is that you activate the virtual environment before creating/working on your project. They don't have to be in the same location. – Alasdair Mar 11 '16 at 09:56