3

It seems it's possible to remove packages (or package versions) for Pypi: How to remove a package from Pypi

This can be a problem if you've completed development of some software and expect to be able to pull the dependencies from Pypi when building.

What are the best practices to protect against this?

static_rtti
  • 53,760
  • 47
  • 136
  • 192

2 Answers2

3

Unnecessary_intro=<< SKIP_HERE

In fact, this is a much deeper problem than just preventing another leftpad instance. In general, practices of dependencies management are defined by community norms which are often implicit. Python is especially bad in this sense, because not just it is implicit about these norms, its package management tools also built on premise of no guarantee of dependencies compatibility. Since the emergence of PyPI, neither package installers guaranteed to install compatible version of dependencies. If package A requires packages B==1.0 and C==1.0, and C requires B==0.8, after installing A you might end up having B==0.8, i.e. A dependencies won't be satisfied.

SKIP_HERE

0. Choose wisely, use signals.

Both developers and package maintainers are aware of the situation. People try to minimize chance of such disruption by selecting "good" projects, i.e. having a healthy community where one person is unlikely to make such decisions, and capable to revive the project in an unlikely case it happens.

Design and evaluation of these signals is an area of active research. Most used factors are number of package contributors (bus factor), healthy practices (tests, CI, quality of documentation), number of forks on GitHub, stars etc.

1. Copy package code under your source tree

This is the simplest scenario if you do not expect package to change a lot but afraid of package deletion or breaking changes. It also gives you advantage of customization the package for your needs; on the other side, package updates now will take quite a bit of effort.

2. Republish a copy of the package on PyPI

I do not remember exact names, but some high profile packages were republished by other developers under different names. In this case all you need is just to copy package files and republish, which is presumably less expensive than maintaining a copy under your source tree. It looks dirty, though, and I would discourage from doing this.

3. A private PyPI mirror

A cleaner but more expensive version of #2.

4. Another layer of abstraction

Some people select few competing alternatives and create an abstraction over them with a capability to use different "backends". The reason for this usually is not to cope with possible package deletion, and depending on complexity of the interfaces it might be quite expensive. An example of such abstraction is Keras, an abstraction for neural networks which provides a consistent interface to both tensorflow and theano backends

5. There are more options

More exotic options include distributing snapshots of virtual environments/containers/VMs, reimplementing the packgage (especially if because of licensing issues) etc.

Community
  • 1
  • 1
Marat
  • 15,215
  • 2
  • 39
  • 48
1

You can create your own local PyPI Mirror(where you update, add, remove packages on your policies) and use your local PyPI mirror for future package installations/update. A full mirrored PyPI will consume about 30GB of storage, so all you need is 30-40GB storage space.

There are many ways to create local pypi mirror. e.g How to roll my own pypi?

Saket Mittal
  • 3,726
  • 3
  • 29
  • 49