35

How can I 'embed' a Python library in my own Python package?

Take the Requests library, for instance. How could I integrate it into my own package, the objective being to allow me to run my application on different machines without actually installing Requests on every one, but having it in the same folder as my package?

Is this even possible?

Gunther
  • 2,474
  • 4
  • 31
  • 45
Sean Bone
  • 3,368
  • 7
  • 31
  • 47
  • 1
    Do you want your application to be installable with ``pip``/``easy_install``, or completely standalone? – fjarri Sep 15 '13 at 13:00
  • @Bogdan it's not necessary for it to be installable - I'll usually be running it myself (so I can just copy the folder), but not on all machines will I have permission to install extra modules. – Sean Bone Sep 15 '13 at 13:03
  • @Sean: of course you do. You can always install packages in your home dir with `pip`. I think you shouldn't try to embed third-party packages because it makes dependency management hard. – Fred Foo Sep 15 '13 at 13:09
  • @larsmans the thing is it's not necessarily always gonna be my own home dir :) – Sean Bone Sep 15 '13 at 14:13

4 Answers4

39

If it's a pure python library (no compiled modules) you can simply place the library in a folder in your project and add that folder to your module search path. Here's an example project:

|- application.py
|- lib
|  `- ...
|- docs
|  `- ...
`- vendor
   |- requests
   |  |- __init__.py
   |  `- ...
   `- other libraries...

The vendor folder in this example contains all third party modules. The file application.py would contain this:

import os
import sys

# Add vendor directory to module search path
parent_dir = os.path.abspath(os.path.dirname(__file__))
vendor_dir = os.path.join(parent_dir, 'vendor')

sys.path.append(vendor_dir)

# Now you can import any library located in the "vendor" folder!
import requests

Bonus fact

As noted by seeafish in the comments, you can install packages directly into the vendor directory:

pip install <pkg_name> -t /path/to/vendor_dir
Hubro
  • 56,214
  • 69
  • 228
  • 381
  • I had to add a "sys.path.append(vendor_dir)" (and import sys) for that to work - leaving this as a comment instead of an edit because 3 whole days of python experience :) – jasondoucette Oct 13 '14 at 12:42
  • @jasondoucette: Indeed my answer seems to have been rushed. I'm declaring the paths, but not adding them to `sys.path`. Will edit – Hubro Oct 13 '14 at 12:46
  • Thanks for the answer. Question: why do you need the `parent_dir` and `vendor_dir` vars when `sys.path.append('vendor')` works on its own? – seeafish Mar 15 '18 at 20:53
  • 1
    @seeafish Because that will only work if "vendor" is in your current working directory. – Hubro Mar 16 '18 at 15:56
  • 4
    I'd like to add that you can install pip packages directly into the aforementioned `vendor` directory by running `pip install -t /path/to/vendor_dir` – seeafish Mar 16 '18 at 20:39
  • @seeafish Nice addition, I added it to the answer – Hubro Mar 17 '18 at 13:18
  • So an impure python module contains `.pyc` files? Could you please explain why this method does not work for those types of modules? – Wilfred Oct 01 '19 at 19:56
  • @Wilfred No, `.pyc` files are just [Python bytecode](https://stackoverflow.com/questions/2998215/if-python-is-interpreted-what-are-pyc-files). By "compiled modules" I meant [C/C++ extensions to Python](https://docs.python.org/3/extending/building.html). – Hubro Oct 02 '19 at 09:46
4

If you only need to run your application may be pyinstaller packaging is a better option.

It will create a single bundle with everything that is needed, including Python, to avoid dependencies on the system you're running in.

6502
  • 112,025
  • 15
  • 165
  • 265
  • Thanks for answering! I'll look into it, though maybe it's over the top for the samll scope this is for :) – Sean Bone Sep 15 '13 at 13:51
2

While not a direct answer to your question. You may want to look at setuptools. By leveraging this package distribution mechanism you can describe your dependencies and when your package is "installed" all the dependent packages will be installed too. You would create a setup.py file at the top of your package structure similar to:

from setuptools import setup, find_packages

setup(
    name = 'MyPackage',
    version = '1.0',
    packages = find_packages(),
    ...
    install_requires = ['requests'],
    ...
)

this would be installed by the user

python setup.py install

Requests would be automatically installed too.

AChampion
  • 29,683
  • 4
  • 59
  • 75
0

All of the above answers are correct but the best solution is creating a standard package.

You can refer to this link: https://packaging.python.org/tutorials/packaging-projects/

Reza Roshan
  • 147
  • 3
  • 7