1

In Post-install script with Python setuptools, this answer shows how to make a post-install command.

I want to make a post-install command that checks for a version matchup between sub-packages from a mono-repo.

How can I get a list of packages being installed during a post-install command?

Current Attempt

from pkg_resources import working_set
from setuptools import setup
from setuptools.command.install import install


class PostInstallCommand(install):
    REPO_BASE_NAME = "foo"

    def run(self) -> None:
        install.run(self)

        one_subpkg_name = f"{self.REPO_BASE_NAME}-one"
        another_subpkg_name = f"{self.REPO_BASE_NAME}-another"
        test_subpkg_name = f"{self.REPO_BASE_NAME}-test"
        all_versions: list[str] = [
            working_set.by_key[one_subpkg_name].version,
            working_set.by_key[another_subpkg_name].version,
            working_set.by_key[test_subpkg_name].version,
        ]
        if len(set(all_versions)) != 1:
            raise NotImplementedError(
                f"test package {test_subpkg_name}'s installed versions "
                f"{all_versions} have a mismatch."
            )


setup(
    ...,
    cmdclass={"install": PostInstallCommand}
)

This solution using pkg_resources.working_set errors out, it seems working_set doesn't function at install-time:

        ...
        File "/private/var/folders/41/wlbjqvm94zn1_vbrg9fqff8m0000gn/T/pip-build-env-1iljhvso/overlay/lib/python3.10/site-packages/setuptools/dist.py", line 1217, in run_command
          super().run_command(command)
        File "/private/var/folders/41/wlbjqvm94zn1_vbrg9fqff8m0000gn/T/pip-build-env-1iljhvso/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
          cmd_obj.run()
        File "<string>", line 39, in run
      KeyError: 'foo-one'

Further Information

Why can't we simply pin versions? Long story short we use setuptools_scm's get_version to dynamically synchronize versions housed in a requirements.txt, so we don't have access to the versions up front.

Intrastellar Explorer
  • 3,005
  • 9
  • 52
  • 119
  • Read the documentation for [`WorkingSet`](https://setuptools.pypa.io/en/latest/pkg_resources.html#workingset-objects). It clearly says that the working set is initialized when importing `pkg_resources`, and most likely not kept up-to-date afterwards... Also what you are trying have high chances to fail anyway, it is quite full of bad practices. – sinoroc Jul 25 '22 at 11:17
  • 1
    Can't you pin the dependencies and let the dependency resolution do its job? – sinoroc Jul 25 '22 at 11:18
  • Sadly I can't @sinoroc and it's too long of a story why. Otherwise I wouldn't ask this left field question – Intrastellar Explorer Jul 25 '22 at 18:23
  • 1
    I see. Well, I would suggest doing this in a separate step, outside of `setup.py`, after installation. I do not think this kind of check belongs in `setup.py`. Anyway, typically the `setup.py` is only used (executed) on the first installation (or build) from the _sdist_, and subsequent installations use the cached wheel. -- Maybe there is a way to do this in `setup.py`, but it feels to me like hammering with a screwdriver. -- Have you tried my first hint: move the `import pkg_resources` to after the installation? – sinoroc Jul 25 '22 at 21:00
  • 1
    There are also `pkg_resources.get_distribution(pkgname).version` and stdlib‘s `importlib.metadata.version(pkgname)`, IIRC both not using any caching mechanism under the hood. But I must agree with @sinoroc this doesn't look like the best approach for this use case. – hoefling Jul 27 '22 at 06:59

1 Answers1

1

I'm not sure if going down this path is best practice, but if you really wanted to you could just run a pip freeze after installation, parse the outputs into an array of sets/dicts and do a Set.difference() across the array items.

subpackage_deps = [
 {"foo=1.1.0", "bar=1.0.0"},
 {"foo=1.1.0", "baz=2.0.1"}
]

subpackage_deps[0] - sub package_deps[1]

I, however, don't see the purpose in doing this. Why not let setup tools do its job? If you are concerned about consistency between projects, why not write an automated test and enforce it using continuous integration to ensure parity of dependency versions between sub-projects (ie: by comparing dependency versions accross setup.py)?

References

https://docs.python.org/3/tutorial/datastructures.html#sets

pygeek
  • 7,356
  • 1
  • 20
  • 41