18

Up to now we keep the version number of our python source code in setup.py.

This version gets increased after every successful ci run.

This means the version of central libraries get increased several times per day.

Since the version number is stored in a file in the git repo, every increase of the version number is a new commit.

This means roughly 50% of all commits are not made by humans, but by CI.

I have got the feeling, that we are on the wrong track. Maybe it is no good solution to keep the version number in ci.

How could we avoid the "useless" CI commits which just increase the version number?

How to avoid keeping version number in source code?

Update

We live without manual release since several years. We do not have a versioning scheme like MAJOR.MINOR. And we have not missed this in the past. I know that this does not work for all environments. But it works for my current environment.

We have a version number which looks like this: YEAR.MONTH.X

This means every commit which passes CI is a new release.

After reading the answers I realize: I need to asks myself: Do I have a version number at all? I think no. I have a build number. More is not needed in this context.

(thank you for the up-votes. Before asking this question I was sure that this question will get closed because people will think it is "unclear" or "too broad")

guettli
  • 25,042
  • 81
  • 346
  • 663
  • 2
    Are you sure you need to keep a version number in the repo at all? Each commit that passes CI could be tagged with a (increasing) version number, leaving the contents of the repository unchanged by a version bump. – chepner Mar 29 '19 at 14:22
  • If CI increases the version number by making a new commit, why not let CI append the modification (of the version number) to the last commit with `git commit --amend`? – Roll Apr 06 '19 at 21:00
  • 3
    @Roll Not a good practice because that would cause change of commit hash, and the original author of the commit will find his local git repo goes out of sync with the remote every time the CI run. He cannot simply git-pull to update because there'll be conflict. – hackape Apr 07 '19 at 10:46
  • 1
    You can also use tools like https://pypi.org/project/setuptools-scm/ which sets the version from the git commits and tags. If you are using poetry there's a hack to do the same: https://github.com/mtkennerly/poetry-dynamic-versioning – gabuzo Mar 17 '20 at 15:18

6 Answers6

5

It is a common practice to keep a version number in the source code, there is nothing wrong in that.

You need to separate CI procedures to regular builds, release publishing and release deployment.

Regular builds: run daily or even after each commit, can include static code analysis and automatic tests, check if the code can be built at all. Regular builds should not change the version number.

Release publishing: can only be triggered by explicit manual action by release manager.
The trigger action could be tagging a commit with a new version number, new merge into the release branch, or just a commit that changes version number kept in a special file (e.g. pom.xml). Refer to git flow for example.
Release publishing assigns a new version number (either automatically or manually), commits it into the source code if necessary, builds a binary package with a new version and uploads it to the binary package repository (e.g. Nexus, devpi, local APT repository, Docker registry and so on).

Release deployment: another manually triggered action that takes a ready binary package from a package repository and installs it to the target environment (dev, QA / UAT / staging, part of production for canary deployments or to the whole production environment).

void
  • 2,759
  • 12
  • 28
  • I updated the question: We live without manual release since several years. We do not have a versioning scheme like MAJOR.MINOR. And we have not missed this in the past. I know that this does not work for all environments. But it works for my current environment. This means every commit which passes CI is a new release. – guettli Mar 29 '19 at 13:07
  • 3
    It sounds like you are actually using build numbers and yes, usually people don't store them in the repository. Still there is a need to identify particular installed version and map it to a commit. In that case you can try using repository revision as a build number. Consider using a Git commit hash or refer to this SO answer: https://stackoverflow.com/a/18834279/5802606. – void Mar 29 '19 at 13:59
4

I think you should use git flow. And create a master branch and a develop branch. Every time the CI checks the develop the version number remains the same. Everytime you create a release e.g. merge develop into master, you can increase the version number by CI.

Or have i missing something, but in my Opinion there is no reason that the version number is increased everytime ci runs.

So all in all you better should think about when to "release" changes to a new version!!

Bierbarbar
  • 1,399
  • 15
  • 35
  • I updated the question: We live without manual release since several years. We do not have a versioning scheme like MAJOR.MINOR. And we have not missed this in the past. I know that this does not work for all environments. But it works for my current environment. This means every commit which passes CI is a new release. – guettli Mar 29 '19 at 13:07
4

Premises:

I assume these are the premises under which the solution is discussed.

  1. Currently version number is kept in a git-tracked source file, but you are OK to get rid of it.
  2. No one manually manages version number, nor triggers a release procedure, which includes: (a) increase version number, (b) build from source and (c) store the built result somewhere. These are taken cared by CI, and SHOULD remain that way.

Solution:

  1. Instead of writing to a source file and create new commit, CI simply tag the specific commit that passed CI check, then push the tag to remote repo.
  2. The build script should read the tag of current HEAD commit, and use it as the version number for publishing a release.
  3. Optionally, you might want to use git filter-branch to rewrite your existing git repo history, tag previous release commits for consistency, remove and stop tracking the version number source cile, then get rid of those CI commits.
guettli
  • 25,042
  • 81
  • 346
  • 663
hackape
  • 18,643
  • 2
  • 29
  • 57
  • May I ask which tool do you suggest to determine the new version number? I was unable to find a proper tool with a CLI interface which reads the last tag and increases it according to specific needs (major, minor, patch, prerelease, etc). Closest is `setuptools_scm`, but it lacks the CLI :( – Hodossy Szabolcs Aug 24 '21 at 11:33
  • @HodossySzabolcs I would recommend tailor-made script instead of any ready-made tool. In OP's scenario they just use date-string as version number. I know many ppl use increasing integer as version number too. So it really depends on your need. – hackape Aug 24 '21 at 11:43
4

If the project's kept in a git repo for production use, just use whichever variant of git describe floats your boat, no need to store it in a tracked file because the result identifies the particular history, and you've got that history right there.

If the source is shipped separately, you can use git archive and the export-subst attribute to embed pretty much anything you want in the exported source.

jthill
  • 55,082
  • 5
  • 77
  • 137
4

PS : Being a new user cannot add comment.

support and expand on this answer by @VibrantVivek.

For Continuous-Integration , tagging of repository back is very important and whether you keep it in your code or simply by any other git way , after every successful CI there must be corresponding tag/version.

And if you're having CI tags/version which are not against commit , then something really wrong is at work here.

And +1 for Martin Fowler , here another link for a more detailed article (more or less by same person) https://www.thoughtworks.com/continuous-integration (recommend to read please)

Piyush Mittal
  • 41
  • 3
3

On your first question :

How could we avoid the "useless" CI commits which just increase the version number?

Please se Continuous Integration (CI) is a development practice which verifies each check-in by an automated build , allowing teams to detect problems early.

Having said that, would like to articulate here :

  • From practice : Every commit should build on an integration machine

  • Under how to do it : The CI server monitors the repository and checks out changes when they occur

In simple words, the CI server should enhance the version only and only when there is a new commit and thus making sure every code commit is releasable.

Looks like from OP , that in your area there more (as you said) "useless" commits from CI server.

Based on your CI mechanism, I hope you should/must be able to control it , almost there are ways to handle in every tool we use. (Eg: webhooks in bitbucket, version plugin etc).

So, making sure only after a new commit we have a new version.

Now if you're thinking about those regular nightly integration builds , then read below :

Many organizations do regular builds on a timed schedule, such as every night. This is not the same thing as a continuous build and isn't enough for continuous integration. The whole point of continuous integration is to find problems as soon as you can. Nightly builds mean that bugs lie undetected for a whole day before anyone discovers them. Once they are in the system that long, it takes a long time to find and remove them.

Also you have mentioned : Every commit which passes CI is a new release, thus in a way you're already on true CI.

Despite this, if you're still unable to figure out how you can avoid "useless" commits of version number, then I would suggest to add another question with detail on how your CI mecahnism works and why it is difficult with given conditions. I bet there must be a solution. Also have look on GithubFlowVsGitFlow.

source : Martin fowler's white paper on CI

How to avoid keeping version number in source code?

On this, would like to expand on @void answer as it is said there there:

It is a common practice to keep a version number in the source code, there is nothing wrong in that.

There are projects which have to know the exact version deployed (for some important xy reasons) in such scenarios they keep version in source and HTTP GET API to fetch from deployed code (one way of doing it) to know the version currently deployed on X server.

However it is more on the requirement, suppose for another project there is no such situation then recommended way to keep version is using commit hash / tagging each successful CI build.

You can have more details here :

Hope this helps.

Vivek
  • 895
  • 11
  • 23