I always use pip install --user packagename – it doesn't require sudo.
If you don't have pip yet, or either OS's pip or OS's easy_install doesn't work properly, first install setuptools in user's home directory, and then use its easy_install for pip.
$ wget https://bootstrap.pypa.io/ez_setup.py -O - | python3 - --user
$ easy_install-3.4 --user pip
On the other side, if you decide to use pip inside virtualenv, don't use --user option for an obvious reason.
Doing it the way you described
For example, I want to import module curl that's present in OS.
$ pip3 install --user virtualenv
$ python3 -m virtualenv myvenv
$ cd myvenv
$ source bin/activate
(myvenv)$ python3
>>> import curl
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named curl
There is no such module inside virtualenv. Just to make sure I looked into what's in sys.path.
>>> import sys
>>> sys.path
['',
'/home/username/myvenv/lib/python27.zip',
'/home/username/myvenv/lib/python2.7',
'/home/username/myvenv/lib/python2.7/lib-dynload',
'/usr/lib/python2.7', '/usr/lib/python2.7/plat-linux2',
'/home/username/myvenv/lib/python2.7/site-packages']
Notice that there is no /usr/lib/python3.4/site-packages, just as you wanted, but /usr/lib/python3.4 is present. If that's not okay for you, use --always-copy option when creating a virtualenv.
In my further steps I create symbolic links to curl module and its dependencies.
>>> ^D # Press Control-D
(myvenv)$ ln -s /usr/lib/python3.4/site-packages/curl lib/python3.4/site-packages/
Notice that there is no trailing slash in the second argument (file or directory which symlink points to), but there is a slash in the third argument (where symlink should be created). That's because I create a symlink inside of the directory lib/python3.4/site-packages. Without a trailing slash, it would try to replace the directory.
Let's check if that works.
(myvenv)$ python3
>>> import curl
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/username/myvenv/lib/python3.4/site-packages/curl/__init__.py", line 9, in <module>
import sys, pycurl
ImportError: No module named 'pycurl'
Nope. curl depends on pycurl. We need to go deeper.
>>> ^D
(myvenv)$ ls -1 /usr/lib/python3.4/site-packages/pycurl*
/usr/lib/python3.4/site-packages/pycurl-7.19.3.1-py3.4.egg-info
/usr/lib/python3.4/site-packages/pycurl.cpython-34m.so
(myvenv)$ ln -s /usr/lib/python3.4/site-packages/pycurl-7.19.3.1-py3.4.egg-info lib/python3.4/site-packages/
(myvenv)$ ln -s /usr/lib/python3.4/site-packages/pycurl.cpython-34m.so lib/python3.4/site-packages/
(myvenv)$ python3
>>> import curl
>>> curl
<module 'curl' from '/home/username/myvenv/lib/python3.4/site-packages/curl/__init__.py'>
At last.
Is this what you need?
Personally, I strongly discourage you from doing this because, firstly, this is a perversion.
Moreover, I doubt this method works on everything. The depth of dependencies can be scary.
Secondly, I see no reason in avoiding --system-site-packages that works like a charm in all cases I'm acknowledged about.
Packages installed by pip in user's home directory have higher priority than system-wide packages, thus, they are taken first when doing import.
In virtualenv, they are only imported if virtualenv was created with --system-site-packages option.
Packages installed by pip inside virtualenv have even higher priority.
Honestly, I wrote this answer just to ask, why would you want to mess with all this instead of getting pip to work. I can't yet write comments on SO, so that's why I wrote a complete answer. :)
But you'll also drag in all the other system wide packages that you don't need!
No, you won't. You load a module into memory only when doing import. If you mean the package will be present in virtualenv – once again, it won't. If you had looked inside of myvenv/lib/python3.4/site-packages/ you would have seen that there are only two packages inside - pip and setuptools (the latter one is separated into several directories though). The Python interpreter inside virtualenv simply loads system-wide modules just like an ordinary interpreter does.
P.S. The code above also applies to Python 2.7. Just replace pip3 with pip2, python3 with python2, 3.4 with 2.7, etc.
P.P.S. Instead of symlinking, you can copy the modules with their dependencies into virtualenv to the corresponding directories. This makes sense when using --always-copy, so that your virtualenv becomes portable.
P.P.P.S. If you decide to use virtualenv with --system-site-packages and pip, and, let's say, you want to have requests package newer than the already installed one in /usr/lib, use pip install -I, as described here.
