263

I'm looking for a way to make a virtualenv which will contain just some libraries (which I chose) of the base python installation.

To be more concrete, I'm trying to import my matplotlib to virtualenv during the creation of virtualenv. It can't be installed efficiently with pip or easy_install since it misses some fortran compiler libs. The way I did it until now was to manually copy from:

/usr/lib/python2.7/dist-packages/ to virtualenv_name/lib/python2.7/dist-packages/

However this prevents the manully imported links to be registerd by yolk (which prints all currently available libs in virtualenv).

So, is there a way to do a selective variant of the

virtualenv --system-site-packages
ouflak
  • 2,458
  • 10
  • 44
  • 49
TheMeaningfulEngineer
  • 15,679
  • 27
  • 85
  • 143
  • @foobarbecue I'm the original OP. I currently don't have the testing environment set anymore, so I will accept the answer as soon I I'm able to test the answers. – TheMeaningfulEngineer Dec 23 '13 at 07:31
  • 2
    looks like 12 people have tested for you over the last few months... – foobarbecue Mar 29 '14 at 14:23
  • Any progress on accepting an answer? Kind of looks like there is one that stands out... – Engineero May 24 '18 at 12:48
  • 2
    There is no correct answer. OP asked for selective `--system-site-packages`. Answers suggests using non-selective `--system-site-packages` and then overinstalling some packages locally, what is different and has different implications. – Piotr Jurkiewicz May 18 '19 at 15:39
  • 3
    Hey, it's me, the OP :) It's been a while since I've asked this question, and every once a while I go take a look what the answers are. The most popular answer is a very good workaround and seems to be accepted by a lot of users (based on the votes). However it results in a virtualenv with system packages which isn't the solution. Adding a step to clean the unwanted system packages in some way would be a complete fix. – TheMeaningfulEngineer Nov 09 '20 at 11:26

4 Answers4

289

Create the environment with virtualenv --system-site-packages . Then, activate the virtualenv and when you want things installed in the virtualenv rather than the system python, use pip install --ignore-installed or pip install -I . That way pip will install what you've requested locally even though a system-wide version exists. Your python interpreter will look first in the virtualenv's package directory, so those packages should shadow the global ones.

foobarbecue
  • 6,780
  • 4
  • 28
  • 54
  • 55
    By using -I, you will always reinstall packages, even if they already exist in the systemwide site-packages directory. If you use -U instead, it will install newer versions of packages into your virtualenv, but won't reinstall any packages that are already available in the system with the required version. – Danilo Bargen Feb 04 '14 at 17:09
  • 20
    Do you know if there is any way of 'activating' the --system-site-packages option on a previously-created virtual environment? I would love to avoid the hassle of reinstalling all my local packages! – Gabriel Apr 29 '15 at 04:08
  • 17
    Yes there is http://stackoverflow.com/questions/3371136/revert-the-no-site-packages-option-with-virtualenv – Mark Apr 29 '15 at 14:55
  • I found this answer very interesting, however I had a very weird case where it didn't work. I have a virtualenv with `no-global-site-packages` enabled. But for some reason I do not know there was a package that was beeing used from the global system. Using `pip install -I` for the package from inside the virtualenv didn't work. I finally ended up uninstalling temporarily the package both from the virtualenv and global system, Then I could install it back in the virtualenv and in the system (in that order). So now it works fine. – kstenger Aug 28 '15 at 12:46
  • Thanks for this. A note regarding -I vs -U: Sometimes a project depends on a specific version (or range of versions) of a package, in which case -I with a version selector is the right choice. – kungphu Jan 29 '16 at 09:19
  • 6
    -1 The question asks for making *certain* packages visible, not *all* except for shadowed packages. Whitelisting a limited set would be much safer than whitelisting everything because it allows to guarantee that all packages must exist in the venv except for the explicitly whitelisted ones. – bluenote10 Oct 17 '18 at 11:50
  • @bluenote10 , do you have a simple way to actually do that? (I've also edited my answer to clarify that you only use -I for packages you want -I for. I do recognize that dependencies could make this tricky to get right.) – foobarbecue Oct 23 '18 at 03:57
  • 1
    @foobarbecue Unfortunately there is no nice solution for this. The best work-around I found is [this answer](https://stackoverflow.com/a/14571660/1804173) which suggest to use `--no-site-packages` and symlink entire packages back from the venv to the system site packages. This seems to work well and allows to precisely control what is visible in the venv. – bluenote10 Oct 23 '18 at 15:18
20

You can use the --system-site-packages and then "overinstall" the specific stuff for your virtualenv. That way, everything you install into your virtualenv will be taken from there, otherwise it will be taken from your system.

PythonJin
  • 4,034
  • 4
  • 32
  • 40
schacki
  • 9,401
  • 5
  • 29
  • 32
  • I am not exactly sure what is unclear, but will try: if you create a virtualenv with --system-site-packages option, the system will first try to find packages in your virtualenv, if it does not find it there, it will try to find it in your system python installation. Only if it does not find it there either, it will raise and ImportError. – schacki Jan 29 '13 at 08:52
  • 7
    The only problem with this approach is that you now have all system libraries. Where you might want very tight control over what is installed. – Dwayne May 31 '13 at 13:20
  • 7
    What's unclear is how this is actually possible! If you do pip install myprog in a virtualenv created with --system-site-packages where myprog exists in the system, it won't "overinstall" myprog. It will just find that myprog exists, and say "Requirement already satisfied." So... what did you mean? – foobarbecue Oct 18 '13 at 18:50
  • To 'overinstall' packages that already in system site-packages, in the virtualenv, run pip with `--force-reinstall` – Gibrahh Aug 20 '20 at 01:52
1

I am late to the game using python.3.8 and pip3 on Ubuntu 20.04.

The ONLY way to get rid of the annoying .local install for me was to set an environment variable (bash):

export PYTHONNOUSERSITE="true"

This does not need to be "true" anything will work. I would not go for a 0. ;-)

ouflak
  • 2,458
  • 10
  • 44
  • 49
Stefan
  • 11
  • 1
-5

Install virtual env with

virtualenv --system-site-packages

and use pip install -U to install matplotlib

gvlasov
  • 18,638
  • 21
  • 74
  • 110
Eder Martins
  • 15
  • 1
  • 1