1

Can a whl file aggregate other whl files?

I realize you can create a family of related whl files, and have them reference each other by referencing the dependent modules in install_requires (https://docs.python.org/3/distutils/setupscript.html)

However, what I'm interested in is aggregating some of those dependencies inside the whl file.

I'm building a 'component' based (charting) system in python - different teams can independently develop these 'chart' components and pluging them into the overall system. The independence of these components implies they maybe developed over different lifetimes and have different versions of shared code (common modules).

Thats why I wish to have these modules AGGREGATED inside each individual chart component, rather than referenced (referencing would cause problems if two chart components dependent on differing versions of common components).

So this question is both a whl file packaging question (straight-forward way to combine) and a question about the abilities of pythons module system to support the scenario of aggregating modules so that they dont get registered in the global sys.modules object, but just used inside a given module (chart component in my case).

anthony sottile
  • 61,815
  • 15
  • 148
  • 207
lewis
  • 1,116
  • 1
  • 12
  • 28
  • @lewis Why would you want to do that? Considering there are ways via setup tools to ensure to `install_requires` includes whatever you want for your package, why this? Because if it's versioned, then it's already released and is available in PyPI. All you have to do is refer the same version in your `install_requires`. Those versions won't change as per your doubt: `they maybe developed over different lifetimes and have different versions of shared code ` – Nagaraj Tantri Sep 07 '20 at 23:22
  • the python module system isn't really designed for npm-style multiple-versioned things. you could vendor a module as a submodule of your package (but this often requires rewriting the internal imports in that module and can be pretty fragile). you might be interested in zipapps, but that's more designed for standalone applications than it is for libraries – anthony sottile Sep 10 '20 at 05:58
  • @AnthonySottile You are correct, but eventually there are ways to isolate the dependency clash via virtuaenv if we are seriously considering the same through versioning, assuming you have a version X of your app, you can always have a library of version A for that and run it and version Y of your app, you can always have a library of version B. This whole whl importing whl is not seemling possible – Nagaraj Tantri Sep 10 '20 at 23:26
  • @ukBaz yes, I've read the documentation on packages. – lewis Sep 15 '20 at 22:25
  • @NagarajTantri the reason to want to aggregate, is because I could have conflicts. Imagine I'm trying to create a complicated modular system, where different people work on different components in different time frames. And image some users combine those things (on their own timeframe). A concrete example may read better - so assume the components are CHART_TYPES (scatterplot, barchart etc). – lewis Sep 15 '20 at 22:28
  • @NagarajTantri (continued). Now the scatterplot chart developers use one version of some component (say the json parser, or some imaging component). And teh barchart team works 6 months later, and selects differnt versions of the same components. – lewis Sep 15 '20 at 22:29
  • @NagarajTantri (continued) - Assume these components (charts) are black boxes. And I want to build a tool that shows a bunch of charts, some scatterplots and some barcharts. What version of the components that are common to both scatterplot and barchart do I load? The version compatible with scatterplot or the version compatible with barchart? If they AGGREGATED their dependencies I wouldn't need to choose, nor to worry. – lewis Sep 15 '20 at 22:31
  • @lewis This might be difficult to achieve with whl being aggregated for whl as much I have explored it. This is however maybe achieved via installing the package locally to a specific directory https://stackoverflow.com/questions/2915471/install-a-python-package-into-a-different-directory-using-pip and creating a version again from there. Not tried, but something which you can explore – Nagaraj Tantri Sep 16 '20 at 05:13

0 Answers0