31

I'm looking at migrating from TFS (Team Foundation Server) to Git, but can't find anything matching TFS' support for gated check-ins (also called pre-tested or delayed commits).

Atlassian Bamboo has no support for gated check-ins. TeamCity does support it ("delayed commits" using their terminology), but not for Git. Using Jenkins by itself or Jenkins+Gerrit has huge drawbacks and doesn't come close to the gated check-in functionality in TFS. (Drawbacks explained by the creator of Jenkins himself in this video: http://www.youtube.com/watch?v=LvCVw5gnAo0)

Git is very popular (for good reason), so how are people solving this problem? What is currently the best solution?

  • 1
    As far as I understand, it is a different mentality. I come from the TFS world as well and I am getting up to speed on Git, but from my research, I have not found a feature that matches gated check-ins. I will keep track of this since I am interested as well. – AdamV Sep 18 '12 at 20:28
  • 1
    Why are you moving to GIT? You can push from your local Git repo to TFS and get all of the benefits of DVCS locally or as a team and then push to TFS for a "matter-of-record" for central storage... kind of like having GitHub bit it is TFS... – MrHinsh - Martin Hinshelwood Sep 22 '12 at 21:46
  • related: https://stackoverflow.com/questions/31681746 – tkruse Jul 26 '18 at 10:12

6 Answers6

16

We have just started using git and have implemented pretested commits using workflows (I finished testing this just today).

basically each dev has a personal repository which they have read/write access. The build server TeamCity in our case, builds using these personal repositories, and then if successful pushes the changes to the 'green' repository. Devs have no write access to 'green', only TeamCity build agents can write to that, but devs pull common updates from 'green'.

So dev pulls from 'green', pushes to personal, TeamCity builds from personal, pushes to green.

This blog post shows the basic model we are using, with GitHub forks for the personal repositories (using forks means that the number of repositories doesn't get out of hand and end up costing more, and means that the developers can manage the personal builds, as they can fork and then create the team city build jobs to get their code pushed to 'green'):

enter image description here

This is more work to set up in TeamCity as each developer has to have their own build configuration. Which actually has to be 2 configurations as TeamCity seems to execute all build steps (including the final 'push to green' step) even if the previous build steps fail (like the tests :)), which meant that we had to have a personal build for the developer, then a another build config which was dependent on that, which would just do the push assuming the build worked.

Sam Holder
  • 32,535
  • 13
  • 101
  • 181
  • 1) Is the personal remote repo actually different branches in a single repo, or do you have multiple repos? 2) How is the "if personal build succeeds, then push changes to green" part done in TeamCity? –  Sep 19 '12 at 07:12
  • 1
    each personal remote is a separate repository, forked from the 'green'. The push to green is done by invoking a new dependent build configuration in TeamCity which has a single step, which is a powershell script which simply executes the 'git push green'. The build has to be done by an agent side get and not a server side get (otherwise the .git directories won't be checked out and so the raw git commands won't be executable) – Sam Holder Sep 19 '12 at 07:59
  • I'm setting this up in TeamCity now. I see the problem where I can't execute the raw git command 'git push green' because there is (or seems to be) no git repo on the TC build server. How did you solve this? –  Sep 19 '12 at 08:50
  • 3
    you have to agent side checkouts. This adds a fair bit of additional complexity, as you now can't use password authentication and need to use ssh. [This blog post](http://devlicio.us/blogs/mike_nichols/archive/2010/06/12/using-team-city-with-git.aspx) and the github page on SSH helped me. – Sam Holder Sep 19 '12 at 09:00
  • I guess I could run git clone every time? Might be slow though. Another alternative is to set up permanent cloned repositories on the build agents? Which would mean I would have to run a command on every build agent for every new repo. –  Sep 19 '12 at 09:06
  • No the agents handle it all for you, and it works fine. you just need to change to use agents and then jump through the security hoops associated with that. The linked blog post is a good starting point. the one thing to watch out for is that the build agents may hang if they are waiting for confirmation for access to github with the SSH key. the solution to that is the StrictHostKeyChecking no in the config file for the agents, but you might also need extra config in there if you need to tunnel SSH through the HTTPS port (as we had to to avoid firewall issues) – Sam Holder Sep 19 '12 at 09:15
  • How does your merge command look? Are you just running 'git merge testedBranch' on the green repo? –  Sep 19 '12 at 09:26
  • I'm not doing a merge (and you don't want to) I just do 'git push --all green' (although I believe the --all is superfluous now as TeamCity only checks out the relevant branch). If the push isn't a fast forward then it fails and the dev has to pull from green and merge the commit and push to personal again. – Sam Holder Sep 19 '12 at 09:39
  • Sorry that should have been 'git push --all origin' – Sam Holder Sep 19 '12 at 09:49
  • I'm a bit new to git and CI, so I'm not following everything. How does 'git push --all origin' push to Green Repo? How should I configure repository to set 'Green Repo' as origin for TeamCity? I understand developer can set his origin to his 'Personal Remote Repo', but how can I do this for TeamCity? Sorry if it's a silly question and don't hesitate to redirect me to git's basics, if needed. – Simon Dec 17 '12 at 13:36
  • @Simon in team city we have a template for a developers personal build. Our pull repo url is set up like this: http://github.com/%GithubUserName%/OurRepo.git, and the push url is set like this: http://github.com/CompanyName/OurRepo.git then when we create a build configuration from this template for a particular developer we specify the build parameter GithubUserName as that devs github user name and it will pull from that users repo. Then the pull happens from the devs repo and temcity sets the push url to be the green repo by default. – Sam Holder Dec 17 '12 at 13:47
  • why the multiple personal remote repos per developer? why don't all share the same remote repo? With GIT each Pull request can be configured to run a build on the remote branch. On a green it can then be merged into the master. Why did not use this approach over the one you are using currently? – user20358 Dec 29 '15 at 09:26
  • We didn't use that because that model allows developers to make any change they like in master, with no checks. The single repo per developer means that no developer has the right to push to the main repo, only the build agent has this ability. This means that the developers **can't** break the build. – Sam Holder Dec 29 '15 at 09:29
  • Not disputing your approach. You probably know what works best for your environment, but to my mind that seems a bit of overkill. The same can be achieved with the raising a pull request on the developers own branch. Any functionality being worked on by the developer needs to be in their own branch locally and before they publish their local branch to the remote repo under a corresp. branch they need to first rebase with remote master to get the latest stuff in - ensure it builds locally - then publish to remote - raise a PR - when it builds green then it gets merged to master. – user20358 Dec 29 '15 at 13:46
  • in case someone mistakenly merges a broken build to the remote master the remote can then be hard reset to a previously good build commit. – user20358 Dec 29 '15 at 13:48
  • Also with GIT, no one should be checking in to master. The beauty of GIT is to think of branches as personal work spaces locally or on the remote for everyone else to peek into and collaborate and when done, then merge with master. – user20358 Dec 29 '15 at 13:50
  • We could had chosen to work like that l suppose, but the hard reset is a fix for a problem that never occurs with the approach we used. It also means we can collaborate on shared feature branches in each traders repo and pull request between them. It also means that the main repo is not cluttered with branches from every dev, each devs branches are only in their repos. I don't see any down side to the one repo per dev approach, this is the model you would use in an open source project, and there are no massive benefits to sharing a single repo IMHO – Sam Holder Dec 29 '15 at 14:09
9

Check out Verigreen - A lightweight, server side gated check-in system. It verifies each commit before it finds its way into the branches the system protects. Verigreen will not allow any failed CI commit to break the integration, release, or any branch that need be protected. Moreover – it's a free, open-source project.

How it works: Verigreen intercepts check-ins and runs verification in an ad-hoc branch - so that in case of failed commit, only the relevant developer is affected.

  • A pre-receive hook intercepts and creates an ad-hoc branch of the code.
  • Verification is run via a Jenkins job. The verification job content is fully configurable.
  • The verified code is merged back into the protected branch whereas a failed commit is blocked with a notification sent to the developer.

enter image description here

Decisions are made based on the following flow:

Verigreen - Basic Flow

For more information, please see the wiki or Verigreen.io site

soninob
  • 428
  • 11
  • 22
  • Is it possible to do add "Code Review" to the chain? – PiotrKowalski Apr 05 '17 at 10:30
  • 1
    Verigreen is using Jenkins to execute a job. You can decide what you want to be on that Job (including tests that will check the qaulity of the code). As one said: "Verigreen supplying the door frame the customer supllies ther door" – soninob Apr 26 '17 at 13:14
4

I think that after October 23, 2013 the answer can be - Automatic Merge in TeamCity.

Stanislav
  • 4,389
  • 2
  • 33
  • 35
0

git has different philosophy - commits are easy, commit as you wish. If something wrong, you can change it later. And merges are easy. So, you could organize a appropriate workflow, e.g. developer(s) could commit in a separate branch(es). When a branch is tested, it could be merged into a main branch.

kan
  • 28,279
  • 7
  • 71
  • 101
  • 9
    The problem with this approach is that what guarantees the merge with the main branch is not going to break the build? Pre tested commits ensure that any changes which break the build should not make it to the main branch/repository which should always be deployable. – Sam Holder Sep 18 '12 at 20:51
  • @SamHolder If merged (pushed) commit fails the build, it could be undone and do not attempt to merge the branch again and leave it to a human to resolve. – kan Sep 18 '12 at 21:46
  • 8
    I'm pretty sure it makes sense to have a repo/branch where no broken code is allowed to get in - to be able to deploy at any time if nothing else. –  Sep 19 '12 at 07:15
  • @arex1337 The continuous integration server could have its own repo/branch as a sandbox, after successful build it could push into the repo/branch where no broken code. Or usually a good practice to create a tag which will mark the good revision by a human friendly name. – kan Sep 19 '12 at 09:00
0

Why not use TFS as the central repository and make use of GIT as a local DVCS solution?

This would allow you to build and commit locally and then push what you want to the TFS server and do a gated build.

Sometimes it is good to have the best of both worlds...

Rami A.
  • 10,302
  • 4
  • 44
  • 87
MrHinsh - Martin Hinshelwood
  • 23,409
  • 5
  • 46
  • 69
  • I actually recommend the opposite. Using TFS as the tool and letting Git be handle the version control. I have setup a repository with a pre-receive hook that passes the $newSha to the TFS repository. If the build passes then a powershell script updates the git repository .git/refs/heads/branchname with the $newSha that had built successfully. Therefore only committing builds that are successful. – Eric Mar 17 '16 at 13:36
0

With VS Team Services (fka Visual Studio Online) and TFS 2015 or newer you can use branch policies requiring a passing build for a pull request as a gated checkin workflow with Git.

Buck Hodges
  • 3,342
  • 1
  • 19
  • 20
  • I need to create a pull request and in case the build succeed, complete the pull request. Can't the second step be done automatically if the build succeed? Thanks. – eskadi May 24 '16 at 11:56
  • We don't currently have the ability to have the pull request automatically complete if the build is successful. It's something we want for ourselves (we use branch policies requiring a build and code reviews), and I hope we get to it soon. – Buck Hodges May 28 '16 at 14:04