7

I develop a Python library using Gitflow development principle and have a CI stage for unit testing and package upload to a (private) PyPI. I want to consume the uploaded package for testing purposes before I merge it back to the integration branch.

Other package managers (and popular tools) allow for version identifiers that contain feature branch specific identifiers, such as 1.2.3-my-feature-alpha.1 in compliance with SemVer. However, PEP-440 forbids the use of such versioning schemes and twine even rejects such uploads.

What is the correct (or a workable) approach to name Python package versions if wanting to create such pre-releases (that potentially can happen in parallel with multiple feature branches) without version identifier conflicts in compliance with PEP-440?

sh0rtcircuit
  • 445
  • 3
  • 13

2 Answers2

5

i am too thinking about this.

Although not intended for this use, you could misuse the local version identifier noted in the PEP-440, see: https://www.python.org/dev/peps/pep-0440/#toc-entry-5

in your case it would be 1.2.3+my-feature.alpha.1

releases with local version identifiers are ignored when querying releases with a compatible public version identifier like 1.2.3, but could still be queried when directly selecting a release with a local version identifier 1.2.3+my-feature-alpha.1

my comment is not a recommendation, it just mirrors the thoughts i had over the same problem.

wobu
  • 196
  • 1
  • 9
  • 1
    Just to add on your thoughts, I believe it would be more accurate to use **both** pre-release and local identifiers together i.e `1.2.3-dev0+my-feature-alpha.1`. This still lets tools consider it a pre-release and the local identifier helps us to pinpoint the specific version we'd like for integration. It was disappointing to discover that PEP-440 decided to lock down pre-releases to 4 distinct values and I totally agree that repurposing the local identifier for this semantics is wrong – mati.o Dec 30 '21 at 16:34
  • @mati.o I believe the correct syntax is `1.2.3.dev0+my-feature-alpha.1` – Ievgen Goichuk Apr 19 '23 at 09:55
  • @IevgenGoichuk Almost! I've dig deeper into PEP 404 and the local identifiers should be limited to alphanumerics and `.`, so `1.2.3.dev0+my.feature.alpha.1` will be more standard. I will update my answer too – mati.o Apr 28 '23 at 12:30
2

So recently we've begun getting CI failures because a newer version of pip started enforcing the standard in a strict manner. So after digging into https://peps.python.org/pep-0440/#local-version-identifiers We were able to implement the following strategy:

Main Branch

The main branch yields a package with a regular semantic version e.g: 1.2.3.

Develop Branch

This branch gets 1.2.3.dev1, note the 1 instead of 0 in the suffix, I'll explain that in a moment.

Feature Branch

Any feature branch coming from dev, say feature/improve-logs gets 1.2.3.dev0+feature.improve.logs.

The local identifier, as stated in PEP-440:

To ensure local version identifiers can be readily incorporated as part of filenames and URLs, and to avoid formatting inconsistencies in hexadecimal hash representations, local version labels MUST be limited to the following set of permitted characters:

  • ASCII letters ([a-zA-Z])
  • ASCII digits ([0-9])
  • periods (.)

Local version labels MUST start and end with an ASCII letter or digit.

dev1 > dev0

Any developer that wishes to use the artifact from dev, states his requirement as follows:

my-library==1.2.3.dev1

This way, no other feature artifact ending with dev0+something will sneak accidentally into his environment. It's not uncommon for us to have 2 features being developed at the same time.

Whereas, a developer that wants to test a specific artifact created from a feature branch will state the full name:

my-library=1.2.3.dev0+feature.improve.logs

Tagging

The whole tagging process happens in the CI, we have a plain text VERSION file to which we append the right suffix after formatting it (replacing / to . etc...)

mati.o
  • 1,398
  • 1
  • 13
  • 23
  • Thanks for sharing your workflow! How are you able to upload the "+local" version identifiers? Isn't it usually rejected by common package repositories? – sh0rtcircuit May 03 '23 at 09:29
  • We use Nexus, and its PyPi Registry absolutely doesn't care about the versions. The only reason we had to switch version schemes is to be compliant with `pip install` which started to enforce it strictly – mati.o May 03 '23 at 10:45