1

We develop enterprise software and we wish to promote more code reuse between our developers (to keep this problem simple, let's assume all .NET). We are about to move to a new VCS system (mostly likely mercurial) and I want to have a strategy in place for how we will share libraries.

What is the best process for managing shared libraries that meets the following use cases:

  1. Black Box - only the public API of the library is known and there is no assumption that consuming developers will be able to "step into" or set breakpoints into the library. The library is a black box. Often a dev does not care about the details, just give me the version of the lib that has always "worked".
  2. Debug - the developer should be able to at least "step into" the library during development. Setting breakpoints would be a bonus too.
  3. Parallel Development - while most likely the minority, there are seemingly valid use cases for developing the library in parallel with the consuming application. Often the authors of the library and component are the same developer. For better or worse, the applications and libraries can often be tightly coupled. Being able to make changes and debug into both can be a very productive way for us to develop.

It should be noted that solving 3, may implicitly solve 2.

Solutions may involve additional tools (such as NuGet, etc.).

codyzu
  • 541
  • 3
  • 18

2 Answers2

1

By sharing libraries, you must distinguish between:

  • source dependencies (you are sharing sources, implying a recompilation within your project)
  • binary dependencies (you are share the delivery, compiled from common sources) and link to it from your project.

Regarding both, NuGet (2.0) finally introduced the "Package Restore During Build", in order to not commit to source control whatever is build in Lib or ExternalDependencies folder.

NuGet (especially with its new hierarchical config, NuGet 2.1) is well suited for module management within a C# project, and will interface with both git and Mercurial.

Combine it with the Mercurial subrepos, and you should be able to isolate in its own repo the common code base you want to reuse.

Vadim Kotov
  • 8,084
  • 8
  • 48
  • 62
VonC
  • 1,262,500
  • 529
  • 4,410
  • 5,250
  • I agree 100% with your distinction. I think binary dependencies are easily managed with NuGet. The problem comes with source dependencies. Specifically, subrepos are a [feature of last resort](http://mercurial.selenic.com/wiki/Subrepository) and add some [risks](http://stackoverflow.com/a/11639890/880509). Also, switching between source and binary dependencies becomes a hassle. When a lib gets mature, I may switch that dependency to binary, but sometime in the future switch it back to source to develop it again. So, is there any way to blur this distinction? – codyzu Oct 25 '12 at 11:52
  • @LordZ subrepo is a feature of last resort because Mercurial says it is. In my experience, it is a dependency management technique like others. It may have its issue in the way Mercurial has implemented it (and its not better with Git: http://stackoverflow.com/questions/12078365/how-exactly-does-git-submodule-work/12078816#12078816), but it still is a good solution. – VonC Oct 25 '12 at 11:59
  • good point... would you simply include the source dependencies as subrepos of the application repo or use the [shell](http://mercurial.selenic.com/wiki/Subrepository#Use_a_thin_shell_repository_to_manage_your_subrepositories) method and have a parent repo that has no code and just manages the relationship between the subrepos? The later approach, while safer, adds some overhead to creating a new application and decouples the relationship between repos with code (which seems to conceptually be a disadvantage). – codyzu Oct 25 '12 at 12:20
  • @LordZ in general, when dealing with dependencies, the shell approach is simpler in order to flatten the hierarchy of dependencies and to force the project to use *one* version of a given subrepo. – VonC Oct 25 '12 at 12:59
1

I have 2 possible solutions to this problem, neither of which seems ideal (and therefore why I posted the question).

  1. Use the VCS to manage the dependencies. Specifically, use mercurial subrepos and always share by source.
    • Advantages:
      1. All 3 usecases are solved.
      2. Only one tool is required for source control and dependency management
    • Disadvantages:
      1. Subrepos feature is considered a feature of last resort by Mercurial developers and from experimentation and reading has the following issues:
        • Tags cannot be easily or atomically applied to multiple repos.
        • Root/Shell repos are inherently fragile (can be broken if the pathing to subrepos changes). Mercurial developers suggest mitigating this issue by including no content in the shell repo and only use it to define (and track the revision) of the subrepos. Therefor allowing a dev to manually recreate a moment in time even if the subrepo pathing is broken.
        • Branching cannot cross repo boundaries (most likely not a big issue as one could argue that branches should only occur in a given subrepo).
  2. Use Ivy or NuGet to manage the dependencies. There are two ways this could work.
    1. Dependencies/Packages can simply contain official binaries. A build server can be configured to publish a new dependency/package into the company repository when a developer submits a build for new version. This solves case 1. Nuget seems to support symbol packages that may solve case 2. Case 3 is not solved and leaves developers in that case out to dry and come up with there own solution (there is basically no way to commit applications to the VCS that include dependencies by source). This seems to be the traditional way that dependency management tools are used.
    2. Dependencies/Packages can contain a script that gets the source from mercurial. The script could be automatically executed when the dependency/package is installed. Some magic has be performed to have the .NET solution include the reference by project (rather than by browsing the filesystem), but in theory this could happen in the NuGet install script and reversed in the uninstall script.
      • Switching between "source" and "binary" dependencies seems to be a manual step. I would argue devs should switch to binary dependencies for releases and perhaps this could be enforced on the build server when creating a release. This further complicated by the fact that the VS solution needs to be modified to reference a project vs a binary.
      • How many source packages exists? Does every binary package contain the script to fetch the source that it was built with? Or do we create separate source packages that use the install script magic to get the source? This leads to the question is there a source package for every tag in mercurial? Every changeset? Or simply 1 source package that just clones and updates to the tip and leaves the dev to update to a previous revision (but this creates the problem of knowing what revision to update to).
      • If the dev then uses mercurial to change the revision of the source, how can this be reflected in the consuming application? The dependency/package that was used to fetch the source has not changed, but the source itself has...
Vadim Kotov
  • 8,084
  • 8
  • 48
  • 62
codyzu
  • 541
  • 3
  • 18
  • Didn't see your answer right away. Good points, even though I tend to combine the two points, and recording the exact configuration I use in version control. +1 – VonC Oct 25 '12 at 13:00