Git submodules not updating in Jenkins build

Question

I have a submodule in a project in Jenkins. I've enabled the advanced setting to recursively update submodules.

When I run the build, I see that the workspace has the files from the submodule. The problem is, it seems to be the first revision of the submodule. When I push changes (repository hosted on GitHub) Jenkins doesn't seem to update the submodule to get the right changes. Has anyone ever seen this?

VonC · Answer 1 · 2018-06-26T19:41:12.897

109

Note that the Jenkins Git plugin 2.0 will have "advance submodule behaviors", which should ensure proper updates of the submodules:

As commented by vikramvi:

Advanced sub-modules behavior > "Path of the reference repo to use during submodule update" against this field , add submodule git url.

Owen B mentions in the comments:

For the authentication issue, there's now a "Use credentials from default remote of parent repository" option

Seen here in JENKINS-20941:

edited Jun 26 '18 at 19:41

answered Aug 05 '13 at 12:03

VonC

1,262,500
529
4,410
5,250

6

But how? Can you also provide detailed steps, which options to choose? Thanks. – zavié Feb 19 '14 at 12:42
8

@zavié I think you should choose "Advanced sub-modules behavior" and then check the checkbox "Recursively update submodules" that will appear, and click Save. – KajMagnus Jun 08 '14 at 03:26
9

This doesn't quite work if you're using a private repository. – Erik Aug 27 '14 at 19:18
1

Worked perfectly for me with a private repo – davegallant Oct 22 '15 at 13:47
3

This only works if your repo doesn't require authentication to read your git submodule. Jenkins bug. – Ernst Kuschke Jun 09 '16 at 07:31
@ErnstKuschke Is there any issue filled for that bug in the Jenkins bugtracker? – VonC Jun 09 '16 at 07:33
Advanced sub-modules behavior > "Path of the reference repo to use during submodule update" against this field , add submodule git url – vikramvi Oct 06 '16 at 16:39
1

@vikramvi Thank you. I have included your comment in the answer for more visibility. – VonC Oct 06 '16 at 20:45
Would be awesome, to have a screenshot, where the right option is selected (for people like me, following screenshots first, before reading anything xD) – d4Rk Feb 27 '17 at 09:39
thanks. it saved a lot of my time. easy to fix after reading your answer. – Alok C Mar 06 '17 at 04:52
You can just check "Recursively update submodules" in Advanced sub-modules behaviours – Afzal N Dec 05 '17 at 16:01
For the authentication issue, there's now a "Use credentials from default remote of parent repository" option – Owen B Jun 26 '18 at 15:12
1

@OwenB Thank you. I have included your comment in the answer for more visibility. – VonC Jun 26 '18 at 19:41
how to configure branch of submodule – vivek Oct 19 '18 at 17:54
@vivek With `git submodule add -b aBranch...`, as in https://stackoverflow.com/a/9189815/6309. – VonC Oct 19 '18 at 20:56
1

Seems like the first submodule gets the parent keys, but not the second one. Still buggy :( – Matthieu Brucher Jan 27 '19 at 09:03
1

@lunesco That would be a good opportunity for asking a separate question with details about your specific setup, so that we can try and understand what would work in your case. – VonC Sep 03 '20 at 20:49

score 38 · Answer 2 · answered Nov 14 '14 at 20:14

38

This is covered in the Git Plugin documentation on the Jenkins site under the section: Recursive submodules.

excerpt

The GIT plugin supports repositories with submodules which in turn have submodules themselves. This must be turned on though: in Job Configuration -> Section Source Code Management, Git -> Advanced Button (under Branches to build) -> Recursively update submodules.

Example

From the configuration screen of your job, in the Source Code Management section, pull the Add button down select "Advanced sub-modules behavior".

Then select "Recursively update submodules":

answered Nov 14 '14 at 20:14

slm

15,396
12
109
124

1

thank you but this didn't work at the time when I tried this (almost 2 years ago) – Ben Nov 17 '14 at 18:40
@Ben - OK, I just tried this and it worked for me. Might be related to your versions. – slm Nov 17 '14 at 19:00
1

This only works if your repo doesn't require authentication to read your git submodule. – Ernst Kuschke Jun 09 '16 at 07:32
@ErnstKuschke - I believe Jenkins can be given an SSH key so that it too can interactive with repos that require auth. – slm Jun 09 '16 at 11:08

score 32 · Answer 3 · edited Aug 12 '18 at 07:46

Are you aware that your Git repository always refers to a particular revision of a submodule? Jenkins is not going to automatically change the revision.

If you want to take a newer revision of the submodule into use, you have to do this in your local Git repository:

cd submoduledir
git pull
cd ..
git add submoduledir
git commit -m 'Updated to latest revision of submoduledir'
git push # Go and watch Jenkins build with the new revision of the submodule

When you do it like this, Jenkins will check out the exact same revision of the submodule during the build. Jenkins does not on its own decide which revision of the submodule to use. This is the fundamental difference between Git submodules and SVN externals.

You might want to read a good reference on submodules, e.g. http://progit.org/book/ch6-6.html.

The ProGit link @sti gave is out of date. I _think_ this is the current equivalent [https://git-scm.com/book/en/v2/Git-Tools-Submodules](https://git-scm.com/book/en/v2/Git-Tools-Submodules) — Stevel, Jun 22 '17 at 20:42

score 19 · Answer 4 · edited May 21 '15 at 08:59

Finally stumbled on a way to do this and it's simple.

The Issue:

The initial clone with credentials works fine but subsequent submodule cloning fails with incorrect credentials.

Automatic advanced sub-module cloning: Source Code Management >> Additional Behaviours >> Advanced sub-modules behaviours: results in credential error.
git submodule update --init in the Execute Shell section also fails with credentials error.

The Solution:

I'm using jenkins-1.574.

Check the Build Environment >> SSH Agent box.
Select the correct credentials (probably the same as selected in Source Code Management section

Update submodules in the Execute Shell section

git submodule sync
git submodule update --init --recursive

Here's a screen shot enter image description here

There's no such checkbox anymore. – adi518 Jul 02 '17 at 12:37 — adi518, Jul 02 '17 at 12:37

score 13 · Accepted Answer · edited Aug 12 '18 at 07:25

13

It looks like I found a solution:

I added a build step to execute the following shell commands:

git submodule foreach git checkout master
git submodule foreach git pull

edited Aug 12 '18 at 07:25

Peter Mortensen

30,738
21
105
131

answered Mar 31 '12 at 06:12

Ben

16,124
22
77
122

After you do those commands, you may need to commit in the superproject though, as the HEAD in your submodules will have been updated. – slacy Sep 27 '12 at 17:34
Hi Ben, could you share your solution with a bit more detail? I want to do the same thing. Also, just to confirm, your solution will git submodule update the submodules of a project into the WORKSPACE, yes? – Kim Stacks Jan 15 '13 at 06:13
there isn't much more detail than that. I just added those 2 lines to my build process and it always pulls the latest version of the submodule. – Ben Jan 15 '13 at 23:28
11

As @sti says in another response here, it seems like you're trying to use Git submodules like SVN externals. Instead of adding these commands to Jenkins, it would be better to commit the proper submodule versions to your main Git repo. Jenkins will then always check out the same version of the submodules when building a particular version of your project. Reproducible builds are a good thing. – Cody Casterline May 01 '13 at 16:58
4

@ben I came across this command that you might find more useful, esepcially if you aren't using the master branch in the submodule `git submodule update --init --recursive` – Corey Scott Nov 20 '13 at 03:12

score 7 · Answer 6 · answered Jan 25 '13 at 17:54

7

If you are using Jenkins Git module, you can set it to "Wipe out workspace before build", this way it will always gets the correct sub module.

answered Jan 25 '13 at 17:54

Amin Y

701
1
9
15

score 6 · Answer 7 · answered Jul 08 '20 at 07:19

6

I am using scripted pipelining with the checkout plugin. If you want the submodules to be the same as in your repository, simply switch off the trackingSubmodules option like this:

checkout([$class: 'GitSCM', branches: [[name: '*/develop']], doGenerateSubmoduleConfigurations: false, extensions: [[$class: 'SubmoduleOption', disableSubmodules: false, parentCredentials: true, recursiveSubmodules: false, reference: '', trackingSubmodules: false]], submoduleCfg: [], userRemoteConfigs: [[credentialsId: '[myCredentials]', url: 'https://git.myRepo.git']]])

answered Jul 08 '20 at 07:19

Jochen Gunzelmann

61
1
5

1

This answer does not rely on a special plugin configuration. – akahunahi Nov 03 '22 at 18:15
best answer! the submodule should be configured via existing jenkins api – Nikita Feb 12 '23 at 09:46

score 2 · Answer 8 · edited Sep 22 '21 at 01:26

2

On "Advanced sub-modules behaviours", check "Update tracking submodules to tip of branch"

edited Sep 22 '21 at 01:26

Gino Mempin

25,369
29
96
135

answered Sep 21 '21 at 09:36

Anjali

21
1

Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers [in the help center](/help/how-to-ask). – Community Sep 21 '21 at 11:01
This worked for me – Bhavesh Sep 30 '21 at 20:05
This should be the accepted answer, works like a charm and with one simple click! – JBoy Feb 28 '22 at 09:00

score 0 · Answer 9 · answered Aug 31 '23 at 11:25

I was unable to get scmGit working with submodules. The build failed at native git call with:

/usr/bin/git config --get submodule..url" returned status code 1:

So my ugly hack solution was initializing the submodule manually:

scmGit ...
sh 'git submodule add --force "<sub-path>" "<sub-name>"'

Git submodules not updating in Jenkins build

9 Answers9

Example

The Issue:

The Solution:

Linked