4

I have a large .NET based project with > 20 separate different solutions that each represent different modules in a larger system. Within each solutions there is a number of projects.

We currently handle the all the solutions in one single Git repo as they kind of belong together and that has worked fine for us so far. Now we however like to start using a CI server to for example build and test on modification. This is however where this start to get hairy. I can't have the CI server build the complete solution but only to build the component that was changed. In SVN we could achieve this with one repo and limit the different configurations of the CI server to listen to different paths - so for example "src/ModuleA" was one build and configuration and "src/ModuleB" was another.

What are my options in Git and what would be considered best practice? What would their pros and cons be? I'd love to be pointed to a larger open source solution with a similar setup as part of the answer.

  1. I guess I could have each module in it own repo? But that a problem when it comes to hosting on GitHub and so on, and as they charge by repo ... Would this otherwise be the best option?
  2. Will Git sub-modules work? Is this the right use for sub-modules?
  3. Do you know of other options in a CI product that helps with this scenario some how?
Riri
  • 11,501
  • 14
  • 63
  • 88
  • I have the same (unsolved) problem with git. Maybe you should read about 'google repo'. Btw, why don't you use SVN? I think SVN (svn:externals) is better for that. Even after years of GIT development – hek2mgl Feb 03 '13 at 14:48
  • 1
    @hek2mgl I have done this with svn:external for multi-repos modular project. It does get messy when you have to branch certain modules and not others because of interdependency issues. – Philip Tenn Feb 03 '13 at 14:59
  • @PhilipTenn Do you have an answer? I would really appreciate it :) – hek2mgl Feb 03 '13 at 15:01
  • 1
    @hek2mgl I wish I did have a good answer. :-) I was just commenting to share the perils I faced with multi-repos modular project and svn:external. – Philip Tenn Feb 03 '13 at 15:06
  • @PhilipTenn Let me know if you have someday! :) – hek2mgl Feb 03 '13 at 15:12

3 Answers3

2

I guess I could have each module in it own repo? But that a problem when it comes to hosting on GitHub and so on, and as they charge by repo ... Would this otherwise be the best option?

That remains the best option, unless those module are too inter-dependent, which means addition hurdle when it comes to manage branching (because you need to remember to branch all or most of all modules at the same time, same for merging back)

If charging is an issue, remember you can have a similar organization in BitBucket repos (which offers free private repos, albeit for a small team)

Once you have each module in their own git repo, you can combine them with subtree or with submodules, as I explain in "Combine a base project that is growing in child projects in git repository except git submodule or subtree merge methods".
git slave can be a simpler alternative as well.

Community
  • 1
  • 1
VonC
  • 1,262,500
  • 529
  • 4,410
  • 5,250
1

While git submodule and git subtree are both good (but different) solutions for sub-component management, they do have some drawbacks

git submodule:

  • you end up with multiple repositories.
  • you have to manage each repository separately.
  • the user interface is at times outright confusing.

git subtree (I haven't worked with it much, so my objections might be unfounded):

  • you have a single history for entire project. While this works good, and there's always a way to separate the history, there's no easy way to compose arbitrary versions of components into a system.

I have devised a method called git components which works in a single repository (like git subtree) and gives you a flexibility of independent component histories (like git submodule).

The method is based on the path checkout: checkout <branch_name> -- . (mind the trailing dot).

The method uses a simple manifest-like bash script and imposes some naming conventions to facilitate component management.

You can find a full description of the method and a toy implementation here:

https://github.com/picrin/git-components

edit

As suggested by VonC, I've tried to combine this with a new git feature available with git 2.5 git worktree. Although 2.5 is not available yet (unless you're coming here from the future), the most recent master has the feature and it seems to work well.

The way I'd suggest using git worktree with git components is by checking out the main working tree to master, and creating a worktree for each component and each version of the system (clientA, clientB, etc.). This way you can work on each component simultaneously without having to stash your work on component 1 whenever somebody interrupts you to do a quick hotfix on component 2.

Community
  • 1
  • 1
Adam Kurkiewicz
  • 1,526
  • 1
  • 15
  • 34
  • 1
    Interesting. +1. I wonder if that checkout-all.sh could benefit from the upcoming feature of multiple working tree of Git 2.5 (http://stackoverflow.com/a/30185564/6309) – VonC Jul 25 '15 at 22:11
  • @VonC thanks for the tip, I've seen the talk of this on the mailing list but I haven't tried it. I've got the most recent master compiled, and it's 2.5.0.rc2, so I'll give it a try and update the answer. – Adam Kurkiewicz Jul 25 '15 at 22:19
1

Another option is to use git sparse-checkout. This enables you to specify which folder(s) within a git repository that you want in your local working directory (e.g. "src/ModuleA") while filtering out everything else.

If your integration tool understands git's sparse-checkout feature, or if you can set up a script to do the sparse-checkout for your integration tool, then this may solve your problem.

Pi Da
  • 359
  • 2
  • 8