5

My problem may be best described with an example.

Assume I have a project "A".

I also have a project "B" which depends on "A".

Another project "C" also depends on "A".

My "main" project depends on "B" and "C". It may also be that it also depends directly on "A".

Looks a bit like the "dreaded diamond of inheritance" when "main" == "D"

dreaded diamond of inheritance

These are my requirements:

  • I'd like to be able to edit the content of projects "A", "B" and "C" in the solution for "main " and submit changes (i.e. I don't want just to include the DLL but also the code). But since "B" and "C" both depend on "A", it should be enforced that they reference the same commit.

  • The projects "A", "B" and "C" will most likely also be referenced by other projects, so I cannot assume ownership of the working directory for project "main".

  • Also it should be possible to sync the repositories for each project with external repositories.

  • The projects "A", "B", "C" and "main" should be

How do I need to set up my repositories to accomplish this?

Community
  • 1
  • 1
Onur
  • 5,017
  • 5
  • 38
  • 54
  • So, there's no way you can configure B's and C's path to D? Sounds to me like that's the easiest fix if you can swing it. – jthill Nov 06 '13 at 15:43
  • @jthill "B" and "C" are used not only by "D" but maybe other projects as well. Each of them may use a different version of "B" and "C". So if I don't wont to check out the correct version of "B" and "C" before compiling project "D" I cannot simply put them in sibling folders and use relative paths. – Onur Nov 06 '13 at 15:47
  • Whoops, sorry, I assumed "A" was the toplevel project. Either way, if you want a common version, have it checked out in just one place and configure everything that needs it to use that. – jthill Nov 06 '13 at 16:13

3 Answers3

6

Do not (ab)use git or any version control system for this.

Use NuGet.

You don't want to directly include other projects. Instead, pack every dependency into a .nuget package and use that for dependency handling. While submodules seems a solution to the problem, in practice, you are much better off using a proper dependency management instead of just including other projects.

TeamCity and other CI systems, even TFS, allow you to automatically build new nuget packages, and TeamCity can also act as a nuget server.

And if you do not want to use a 'real' NuGet server, you can also just use a shared drive somewhere on your intranet.

Wilbert
  • 7,251
  • 6
  • 51
  • 91
  • We use NuGet for integrating external libraries. But I'd prefer not setting up our own NuGet-server (our company policy allows only intranet solutions). I find it also handy to be able to step through the libraries source code. – Onur Nov 04 '13 at 14:00
  • You can still step through the code if you use NuGet, just make the symbols (pdb) accessible. And you can even use a shared drive as 'nuget server', if you don't want to set up one specifically. I have tried using VCS in the way you want before - it just leads to so much more pain that doing it properly using a dependency management system like NuGet.package manager). – Wilbert Nov 04 '13 at 14:28
5

Using the submodules approach, I would recommend a model which includes a list of dependencies, instead of a hierarchy of dependencies:

Create a "parent" repo, in which you "git submodule add" the submodules for D, C, B and A:

parent
  D
  C
  B
  A

Add any symlink (even in Windows) you need for each project to compile (meaning to find the sources for their dependencies right from within there own structure).

The "parent" repo will reference the exact list of SHA1 representing each dependencies at their exact version needed for the project to compile/run at a specific time in the history of the parent repo.

And, as I explain in "true nature of submodules", you can make any modification you want in A, B, C or D, and push to their respective upstream repo.
But you must not forget to go back to the parent repo, add and commit the SHA1 modifications representing the new state of the dependent repos A, B, C, and D.

This is a component approach, which has the advantage of resolving any overlapping dependency: if B and C need a different version of A, the parent repo will have to chose one (and only one) version of A.

To add to the OP Onur's answer:

Dependencies must be relative

Not necessarily: you can add symlink within a submodule, if it needs to see another with a fixed path.

Create the "user" repository

How do I automatically check out the correct set of libraries, e.g. A and B in the setup above

Once you have determined valid starting version for A and B, you can create and push a branch dedicated for their usage in a "parent" context.
Then you make those submodule follow that branch, meaning any git submodule update --remote will checkout the latest SHA1 of that dedicated branch for submodules A and B.

Community
  • 1
  • 1
VonC
  • 1,262,500
  • 529
  • 4,410
  • 5,250
  • I added a new "answer" to make sure I understand what you mean. Feel free to edit it. – Onur Nov 04 '13 at 12:18
  • @Onur I have edited *my* answer to address your comments and questions that you published as an "answer". – VonC Nov 04 '13 at 12:31
  • I updated the workflow for the "user" project. I'm still not sure how I can only use the libraries I need and not all available ones. – Onur Nov 04 '13 at 12:52
  • @Onur are each of your libraries an independent git repo? Then yes. – VonC Nov 04 '13 at 12:54
  • They don't have to be independent repositories if it's not necessary. But I thought they'd have to be if I only want to check out only a subset of them (without using subtree etc.). "Then yes" means, it's possible to use only the necessary libraries? How? – Onur Nov 04 '13 at 13:02
  • @Onur by making sure each library source is an independent git repo: then you can `git submodule add` that repo. – VonC Nov 04 '13 at 13:04
  • Ok, but how do I check them out consistently if I don't include the "parent" repository which contains consistent sets of libraries? – Onur Nov 04 '13 at 13:05
  • @Onur the submodule main characteristic is to checkout a specific SHA1, the same every time. If you go in a submodule and checkout any new SHA1 (or make any new commit), then you go back to the parent repo, add and commit that new SHA1. Meaning the next clone of that parent repo will " check them out consistently". More in http://stackoverflow.com/a/1979194/6309. – VonC Nov 04 '13 at 13:07
  • I know that a submodule creates a consistent version for (in this case) "user" , "A" and "B". But this information is contained within the "user" repository. How can the "parent" project access this information that a specific version of "A" is needed for a new version of "B"? Please see also my updated answer. – Onur Nov 04 '13 at 13:26
  • @Onur ditch "user". Don't try to make n branches, one per user, or "user" folder, or anything "user" related. keep it simple. One parent repo. 4 sub-folders, one per submodules. Simple. – VonC Nov 04 '13 at 13:28
  • It's still not clear to me how the "parent" repository knows which versions of "A" and "B" were changed in lockstep by an arbitrary user project. – Onur Nov 04 '13 at 13:34
  • @Onur If those changes are done in a branch that parent submodules are set to monitor (as in http://stackoverflow.com/a/9189815/6309), then a `git submodule --remote` will detect the new commits and update the submodules accordingly. – VonC Nov 04 '13 at 13:37
  • If I ditch the "from_user/for_user" branches, every "user" project commits to the same branch in a libraries repository? – Onur Nov 04 '13 at 13:55
  • @Onur yes, or more accurately, the branch followed by the submodules in the repo "parent" is the one where "validated" commits have been done for A, B, C and D. You have to agree on an "main" branch per submodule repo, which will be followed by a parent repo. That doesn't prevent each repo to be developed by many developers in many branches. But a "parent" repo should only see *one* branch. – VonC Nov 04 '13 at 13:58
3

Just to make sure I understand your approach:

Things I need to do to set up the library repository:

  1. Create the "parent" repository
  2. Create a submodule for each library
  3. Dependencies must be relative, i.e. if D depends on B the path should look like ../../B/some folder
  4. Each commit in the parent repository represents a valid combination of libraries

Things I need to do to include a chosen set of libraries into a new project:

  1. Create the "user" repository
  2. Create a branch in the required libraries respective repositories (e.g. A and B) like "for_user" and track this for the submodules in the "user" project.
  3. Create a "from_user" branch in the "parent" repository that tracks changes from the used libraries (is it possible to track repository branches on a per-branch base?).
  4. Include the chosen set of libraries, e.g. A and B from the "for_user" branch.
  5. Changes made in A and/or B go to this branch and may be merged into the "parent" master branch if suitable and/or merged into other "from_..." branches from other projects using these libraries. The "parent" project is used to create consistent sets of libraries.

Effectively this means every "user" project that uses these libraries is represented by a branch "from_..." in the "parent" repository and a branch "for_..." in each used libraries repository.

The setup should now look like this

A
   branch "for_user"
   branch "master" (== for_parent)
   <other branches for each project using this library>

B,C,D
   <like A>


parent
   submodule A - tracks branch dependent on the current branch
   submodule B - tracks branch dependent on the current branch
   <other submodules>
   branch "master" (== from_parent)
   branch "from_user"
   <other branches for each project using one of the libraries>

user
   <own stuff>
   submodule A - tracks branch "for_user"
   submodule B - tracks branch "for_user"

Open questions:

  • How do I automatically check out the correct set of libraries, e.g. A and B in the setup above? I could check out all available libraries, but that would somehow eliminate the need for separate sub-modules for each libraries
Onur
  • 5,017
  • 5
  • 38
  • 54
  • Starting to looking good. Just remove the branch "`from_user`" in the `parent` repo. A `parent` repo isn't aware of "users", only of one specific branch it has to follow for the submodules. – VonC Nov 04 '13 at 14:37