1

     What tooling exists that is designed to make it easy (or at least easier) to update a Git repository and its submodule metadata when the source or sources for any submodules contained in that parent repository changes upstream? I've seen advice that suggests using Git hooks or continuous integration and deployment systems to manage things like this in such a manner, but nobody seems to have come up with and fully explained a comprehensive solution for it that just works.

RandomDSdevel
  • 431
  • 6
  • 17
  • 1
    Can you provide more context? Is this to update source on your dev box, or to ensure that a build is made with the new code on a server, etc. Need a bit more flesh on what the need is. – Rob Smyth Apr 03 '18 at 03:05
  • @RobSmyth: Sure. I was leaning more toward wondering how to cover the latter case where a submodule receives one or more updates and one or more of its downstream users want to have these updates' appearance upstream trigger one or more CI/CD build jobs downstream. The former case does seem like it could possibly come up in some situations as well, though, but, at a guess, I think that could be handled by reusing some or all of the same tooling, but just invoking it manually; is that correct? – RandomDSdevel Apr 03 '18 at 19:35

1 Answers1

1

It depends first how the main parent repository is referencing its submodules.
If it has submodules tracking a branch, then a simple command is enough to update a parent repo with new submodule content:

git submodule update --recursive --remote

All you then need to do is execute that regularly (cron job, Jenkins job, ...) and you main repository will always be up to date.

could CI/CD see that a submodule the repository with which it's associated depends on has changed and run this for you if it can?

A job could:

  • get the SHA1 of the gitlink associated to a submodule

    git rev-parse released-1.2.3^{commit}:foo

  • perform the update of all submodules

    git submodule update --recursive --remote

  • compare the SHA1 of the submodule to the original one: if there is any change, the submodule no longer reference the same SHA1 and has had some changes.

VonC
  • 1,262,500
  • 529
  • 4,410
  • 5,250
  • It looks from the content you referenced that I'd also have to record the resulting submodule metadata changes to the affected parent repository with a commit. – RandomDSdevel Apr 03 '18 at 19:47
  • Also, what do you do if you (accidentally on purpose) begin to have one of your parent repository's submodules not track a branch (any longer, if it did before?) I take it that doing the equivalent of `git submodule update --recursive --remote` is rather more complicated in that case? – RandomDSdevel Apr 03 '18 at 19:50
  • 1
    @RandomDSdevel yes, that is called a gitlink, special entry in the index of the parent repo: https://stackoverflow.com/a/19354410/6309 – VonC Apr 03 '18 at 19:51
  • 1
    @RandomDSdevel As long as you want to track the master branch of a submodule, it will, by default, even without explicit tracking: https://stackoverflow.com/a/37924157/6309 – VonC Apr 03 '18 at 19:53
  • Thanks. That's all well and good, but it doesn't _quite_ answer my question: could CI/CD see that a submodule the repository with which it's associated depends on has changed and run this for you if it can? How would one set that up? – RandomDSdevel Apr 04 '18 at 23:55
  • I accepted your answer as the canonical one for this question and awarded you my bounty for it even though you don't exactly provide instructions in the detail I was looking for; what you've put down should be good enough to work with given external reference material. – RandomDSdevel Apr 06 '18 at 19:03
  • 1
    @RandomDSdevel Sorry, I meant to address your question this evening. I have edited my answer to add the necessary steps. – VonC Apr 06 '18 at 19:27
  • Thanks! One last thing, though: what, roughly, would one do if they wanted to invert the chain of responsibility as to what happens where when a submodule gets updated — that is, one wanted to have a repository _notify_ other repositories including it as a submodule of new commits to itself rather than requiring all repositories containing submodules tracking this first repository poll it for new commits periodically? – RandomDSdevel Apr 06 '18 at 22:52
  • 1
    @RandomDSdevel therein would lie the rub: a repo is not aware it is a submodule of an upstream parent repo. So you would need an external ledger mechanism, taking not of which repo is updated, and of which repo includes submodule, reconciling the two sets, and notifying. That ledger would be called by each repo by their own respective post-receive hook (after receiving a commit). – VonC Apr 06 '18 at 23:03
  • Right. Hmm, I think you've just sparked some ideas in me as to how what you suggest could be a _tad_ simpler…; thanks for the pointers! – RandomDSdevel Apr 06 '18 at 23:24