can I make a virtualenv into an egg

Question

I can make a python egg using setuptools: python setup.py bdist_egg. In theory, I should be able to do this for every installed package with a setup.py file. Is it possible to encapsulate an entire virtualenv virtual python environment to a python egg?

Answer 1

I specifically needed to package an arbitrary number of Python packages into one .egg, because the system that I was using accepted only eggs and each of them needed to be listed separately which became very unwieldly when the number of needed packages changes.

An .egg file is just a .zip with a metadata folder named EGG-INFO and a version number in its name. You could basically cd lib/python-x.y/site-packages, then zip -r spaghetti-0.0.1.egg to zip the contents into into spaghetti-0.0.1.egg file, but you do need metadata

If you're lucky and you do not have packages that use entrypoints or other such advanced features, you can just create a directory named EGG-INFO in the site-packages with the following files in it:

dependency_links.txt
entry_points.txt
not-zip-safe
PKG-INFO
requires.txt
top_level.txt

all of them empty besides PKG-INFO that contains the following contents

Metadata-Version: 1.1
Name: spaghetti
Version: 0.0.1

and top_level.txt containing the all top-level package names from your virtualenv, one per line, i.e. if you've installed the namespace package zope.component and sqlalchemy, your top_level.txt should have

zope
sqlalchemy

---

Of course things are not always this simple. For namespace packages on Python 2.7 (such is the case with zope.component) there are some magic .pth entries. For these you need to create empty __init__.pys in the packages, or alternatively list them in the EGG-INFO/namespace_packages.txt; in the case of zope.component, the zope is a namespace package with no __init__.py, so EGG-INFO should have namespace_packages.txt with one line, zope. However namespace packaging in Python 3 should work as-is without this intermediate step.

Likewise if you need to use entrypoints, you need to concatenate the entrypoints.txt from the egg infos from all packages into the entrypoints.txt of your egg.

Correction: you cannot do entry points in this fashion, not without any serious hacks anyway. The distribution name, in this case spaghetti, would be used for all entry points instead of the package name. There is no direct way of circumventing this.

---

Finally, wheel indeed can be considered a format that is superior to egg but they're not compatible and if you can you should use wheel for packaging the virtual environment. But if a system specifically expects to have a file in the old .egg file format it wouldn't work with wheel. Furthermore, an .egg sometimes needs not be installed, it can be used from PYTHON_PATH as it is...

Answer 2

You shouldn't do this for several reasons

Even if you could manage to do this, it wouldn't work as you'd expect. If the user already had one of the packages inside your egg installed, they would clash, because python eggs don't add additional namespaces.

Also, python eggs are kind of on their way out. If you want to create a binary distribution, you should use the newer python wheels.

If you have a project with dependencies, just declare those dependencies in your setup.py and pip will handle installing them. If you have some dependencies that don't exist on PyPi, or you need a very specific legacy version of a package, just include those packages as a subpackage in your package.

/my_package
    __init__.py
    /libs
        __init__.py
        /non_pypi_package
        /legacy_package 

And then in your code, instead of importing them as

import non_pypy_package

you would use

from my_package.libs import non_pypi_package