Where should I put my own python module so that it can be imported

Question

I have my own package in python and I am using it very often. what is the most elegant or conventional directory where i should put my package so it is going to be imported without playing with PYTHONPATH or sys.path?

What about site-packages for example? /usr/lib/python2.7/site-packages.
Is it common in python to copy and paste the package there ?

Answer 1

I usually put the stuff i want to have ready to import in the user site directory:

~/.local/lib/pythonX.X/site-packages

To show the right directory for your platform, you can use python -m site --user-site

---

edit: it will show up in sys.path once you create it:

mkdir -p "`python -m site --user-site`"

Answer 2

So if your a novice like myself and your directories are not very well organized you may want to try this method.

Open your python terminal. Import a module that you know works such as numpy in my case and do the following. Import numpy

numpy.__file__

which results in

'/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site- packages/numpy/__init__.py'

The result of numpy.__file__ is the location you should put the python file with your module (excluding the numpy/__init__.py) so for me that would be

/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site- packages

To do this just go to your terminal and type

mv "location of your module" "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site- packages"

Now you should be able to import your module.

Answer 3

This is something that works for me (I have to frequently create python packages that are uploaded to a private pip repository). elaborating on the comment by @joran on the question.

1. create a "build directory" which is used as a workspace to create packages. any directory of your choice will do
2. Copy your python package dir there, and create a setup.py file. this should help in creating the setup.py correctly.
3. create a virtualenv for the project you are working on. virtualenvs have a bunch of other benefits, I am not going into the details here.
4. create a local dist package python setup.py sdist --format=tar. the package created should ideally be in the dist folder.
5. Install the package on your virtualenv (after activating it). pip install <yourpackage>.tar

you can use pip install --force-reinstall if you need to play around with the libraries more and re-create the dist packages.

I've found that this method works great for me. If you do not need to package the modules for use of other systems instead of just your local, this method might be an overkill

Happy hacking.

Answer 4

If you're developing a module I would recommend to follow this.

import sys
sys.path.append("/home/mylinux/python-packages")

Now any module you keep in python-packages is importable by Python-interpreter.

Answer 5

On my Mac, I did a sudo find / -name "site-packages". That gave me a few paths like /Library/Python/2.6/site-packages, /Library/Python/2.7/site-packages, and /opt/X11/lib/python2.6/site-packages.

So, I knew where to put my modules if I was using v2.7 or v2.6.

Hope it helps.

Answer 6

import folders could be extracted by adding following source code:

import sys
print sys.path

automatic symlink generation example would be:

ln -s \`pip show em | grep "Location"| cut -d " " -f2\` \`python -m site --user-site\`

instead of "em" you may use other package you've "just installed but the python can't see it"

below I'll explain in more details as being requested in the comment.

suppose you've installed python module em or pyserial with the following command (examples are for ubuntu):

sudo pip install pyserial

and the output is like this:

Collecting pyserial
  Downloading pyserial-3.3-py2.py3-none-any.whl (189kB)
    100% |████████████████████████████████| 194kB 2.3MB/s 
Installing collected packages: pyserial
Successfully installed pyserial-3.3

the question would be following - python can't see the module pyserial, why? because the location where the module has been installed isn't the one python is looking at for your particular user account.

solution - we have to create symlink from the path where pyserial arrived to the path where your python is looking for.

symlink creation command would be:

ln -s <what_to_link> <where_to_link>

instead of typing exact location we are asking pip to tell us where it stored modules by executing command:

pip show pyserial | grep "Location"| cut -d " " -f2

instead of typing exact location we are asking python to tell us where it looks for the modules being installed by executing command:

python -m site --user-site

both commands has to be escaped with "`" character (usually on the left of your 1 button for the US keyboards)

in result following command will be provided for ln and the missing symlink would be created:

ln -s /usr/local/lib/python2.7/dist-packages /home/<your_username>/.local/lib/python2.7/site-packages 

or something similar, depending on your distro and python/pip defaults.