7

What is the recommended workflow if I want to pull a git repo into a virtual Python environment? I think it's easiest to:

  1. Create an empty directory
  2. cd into it and git clone <repo>, as this will work only in empty directories
  3. create the virtual environment in a subdir in the same directory
  4. Start the VE and install requirements.txt from the <repo>.
  5. Add the subdir (and other stuff for my eyes only) to .gitignore
  6. work on project as intended
  7. perhaps ask to push valuable/shareable results to the original repo.

Or is there a better way?

Disclaimer: I'm quite a n00b with git, please bear with me. I'm well aware there are multiple similar answers already, however I couldn't find a convincing TL;DR. Perhaps I missed it.

RolfBly
  • 3,612
  • 5
  • 32
  • 46
  • What is the purpose to add VE into git? I assume requirements.txt should be enough usually. – Chuan Mar 12 '19 at 13:50
  • @Chuan The VE isolates the project, ie the clone. In the VE, I have no more, no more, no less, than what's in its requirements.txt. I also have a 'global' Python dev environment (several, in fact) that may have different versions of the libraries required for the clone. – RolfBly Mar 12 '19 at 14:41
  • Sure, understand. But VE should not be added into git IMHO, it should only be used to isolate the working env as you mentioned. – Chuan Mar 12 '19 at 15:34
  • @Chuan I see what you mean. You have the VE outside the clone, right? That means more typing (more typo's). I like to keep things as short as possible. I guess it's a matter of taste – RolfBly Mar 12 '19 at 16:22
  • 1
    FYI, I usually use virtualenvwrapper to manage all my VEs and never commit VEs into git – Chuan Mar 12 '19 at 16:58
  • @Chuan thank you. I didn't know about that. – RolfBly Mar 13 '19 at 13:17

3 Answers3

4

This is what I like to do:

git clone <repo> <repodir>
cd <repodir>
virtualenv .venv
. .venv/bin/activate
pip install -e .

That's it. Now any edits I make will be "live" in the virtual environment. I have a global .gitignore file (git config --global core.excludesfile ~/.gitignore) that includes .venv, so I don't need to update .gitignore in individual repositories.

Note that you don't need to run git clone in an empty directory. Because it creates a new directory, it doesn't care of the current directory is empty or not.

Update

Look, we're in a non-empty directory:

bash-4.4$ ls
file1  file2  file3

We can run git clone here:

bash-4.4$ git clone https://github.com/git/git
Cloning into 'git'...
remote: Enumerating objects: 264438, done.
remote: Total 264438 (delta 0), reused 0 (delta 0), pack-reused 264438
Receiving objects: 100% (264438/264438), 112.69 MiB | 9.84 MiB/s, done.
Resolving deltas: 100% (196057/196057), done.

It runs without a problem and creates a new directory:

bash-4.4$ ls
file1  file2  file3  git
larsks
  • 277,717
  • 41
  • 399
  • 399
  • Hmmm, git for windows won't let me git clone to the root of a drive 'because it is not empty'. Same for Linux git, I believe, see [here](https://www.itheo.nl/how-to-git-clone-into-a-non-empty-directory/) also. – RolfBly Mar 12 '19 at 16:19
  • No, that's absolutely not the same for Linux git. You can't run `git init` in a non-empty directory, but `git clone` doesn't care (because, like I said, it will *create* a new subdirectory). I've added an example in the answer. I would be surprised to see that git under windows behaved any differently. – larsks Mar 12 '19 at 16:20
  • Windows: "This folder contains files. Git can only clone to empty folders". Here's a [screenshot](https://photos.app.goo.gl/ds3iN3EvH4eFCzXJ6) (I hope). – RolfBly Mar 12 '19 at 16:46
  • I see; you're not using the command line. I can't comment on how the GUI operates, but it looks like in that case it is asking you to provide a directory into which it will place the cloned files (from the command line, that might look something like `git clone exisitingdir/`, in which case, sure, the target must be empty. But that's generally not how one operates `git`. – larsks Mar 12 '19 at 16:59
  • The Windows GUI, when you enter the repo url e.g. `https://git/me/foo` and your default target dir is `Z:\`, it creates `Z:\foo` and clones into it. If you first create `Z:\foo` put a VE in it, then attempt to git clone alongside it, git refuses, for the aforementioned reason. – RolfBly Mar 13 '19 at 13:23
  • This works for me, apart from the final command: `pip install -e .` That requires either a `setup.py` or `pyproject.toml` file; the repo I'm cloning doesn't contain either. In my case, it was just a single library I needed to install which worked fine with the classic `pip install` route. – Bobby Jack Jul 11 '23 at 16:27
3

If there's already an existing virtualenv for a project that you want to clone a library into to work with, the following steps will help you:

  1. Activate the project's virtualenv
    source .venv/bin/activate.fish
  2. Uninstall the package you want to have replaced with a git clone:
    pip uninstall <package>
  3. Re-install the package via git:
    pip install -e git+ssh://git@github.com/<org>/<package>.git#egg=<package>
  4. Change to package directory to work on the repository
    cd .venv/src/<package>
jnns
  • 5,148
  • 4
  • 47
  • 74
0

As far as I am awared, it is not recommended to initialize and store your virtual environment in git directory of your project. In this case your env space is exposed to some kind of vulnarabilities. Among them:

  1. If you remove your project directory, you lose your env. So there is no scalability to properly use it for other projects.
  2. You need always keep in mind what is in .gitignore file in order to avoid unwanted distribution of your env.
  3. I don't know any easy way to list all your envs with standard python3-venv. So, probably, if you operate like that, ocassionaly you will struggle to just find what available envs do you have.

Instead, recommended way is to use virtualenvwrapper tool, that is storing all envs in one specific directory, most commonly in ~/.virtualenvs. This tool helps with creating, deleting and switching between your envs. See more in this post.

With all said above, recommended workflow:

  • install virtualenvwrapper
  • create new env with mkvirtualenv $env_name
  • initialize your rep with git clone <rep>
  • run pip install -r requirements.txt under your $env_name environment
TaskerBliss
  • 51
  • 5