1

The situation...

I have several Git repos which are all needed for my build. I'm actively developing code on one, whilst the others contain library code I use. Since each repo is independent, of course they all have different branch names.

For my Jenkins build, I would like to fetch the relevant versions of code from each repo, put them in the correct subdirectories, and build my project. Ideally I should also be able to do a shallow clone (because one of these repos is large), and be able to do a sparse checkout of only the path I want.

The Multiple SCMs plugin seemed to be the ideal tool for the job. However it is not in active development, and I've seen it throwing asserts which other people have talked about.

I've seen this question which answers how to use the Jenkins Pipeline to do this, so I've investigated the Jenkins Pipeline. I quickly found from the Git Pipeline documentation that its support for Git could most charitably be described as "minimal", a statement which is equally true for the rest of the Pipeline concept. (And that's before we get into the nightmare which is replacing a perfectly workable UI with a text-only interface. Maintenance nightmare, much? Ugh!)

I could also set up Git subprojects. I'd rather not have to go down this route as a way to solve the inadequacies of the latest version of Jenkins, but needs must if it's the only solution.

I will say that a solution other than Jenkins isn't really an option, because we have been using Jenkins in our company for some time and we don't really want to have to set up something else.

Vadim Kotov
  • 8,084
  • 8
  • 48
  • 62
Graham
  • 1,655
  • 9
  • 19

1 Answers1

2

I think the git support in Pipeline is not lacking at all. At least not anymore. And the pipeline is extremely powerful. Certainly not minimal. Maybe you are looking at old documentation. The declarative pipeline could possibly still be considered a bit immature, but still very powerful and easy, and usually my default choice unless I need to get a little more crazy.

The documentation you posted for the git build step is just a wrapper of the checkout scm step, that can be used for very simple git operations. That is certainly not the extent of the options available for using git in Jenkins.

In particular, I have a multibranch pipeline job that watches a git repo. When a change is detected, the repo is pulled down, then I pull down a sparse checkout of another repo into a subdirectory, and then pull another full repo down into another directory. I run some build scripts. Zip some things, deploy the zip file, then run some remote ssh processes to do things with the packages on the remote server.

In a declarative pipeline I do this for a sparse checkout: 

dir("package/infra") {
    deleteDir()  //start with a clean directory

    checkout([$class: 'GitSCM', 
      branches: [[name: '*/master']],
      extensions: [[$class: 'SparseCheckoutPaths', sparseCheckoutPaths: [[path: "my/path/here"]]]],
      userRemoteConfigs: [[credentialsId: 'asdf-fdsa-werw5-asjksadf-wlfjsdf', url: 'git@github.com:ABC/DEF.git']]
    ])

}

You can also do Shallow clones, and all manner of complex git behavior.

You may be better off with a second build for your libraries, and store the built artifacts in artifactory, or just archive them in Jenkins. Then bring the artifacts in from artifactory or use the copy Artifacts plugin. to bring them from another job. But every situation is different.

Damon
  • 1,249
  • 1
  • 15
  • 27
Rob Hales
  • 5,123
  • 1
  • 21
  • 33
  • Thanks Rob. So the problem isn't with what the plugin can do, it's the documentation. :/ I'll keep digging then. – Graham Oct 23 '17 at 10:09
  • Note deleting the directory before calling the sparse checkout will cause git to report ```git read-tree -mu HEAD" returned status code 128: stdout: stderr: fatal: Not a valid object name HEAD```. This isn't breaking my build. [This answer](https://stackoverflow.com/a/45886726/734790) explains what is going on and why it isn't a major concern. Resolve it though by not deleting the directory, but adding CleanBeforeCheckout extension eg `extensions: [[$class: 'CleanBeforeCheckout'],[$class: 'SparseCheckoutPaths',sparseCheckoutPaths: [[path: "my/path/here"]]]],` – Damon Apr 23 '18 at 22:30