How to break a large mercurial repository into smaller ones

Question

This is a duplicate of this question, but specpifically regarding mercurial.

I have a massive subversion repository that I have converted to mercurial. It now looks like this:

MassiveFolder/
   .hg/
   bin/
   common/
   projA/
   projB/
   ...
   projX/
   makefile
   README

I would like each subproject to be tracked in its own repository for better performance. Notice that they all build into the same bin folder. I don't think that will be a problem, but I am not sure if I will need to do anything to adjust the paths. It is safe to assume each developer will clone the repositories in the same structure, but if there was some way to make a meta-repository that can ensure that, it would be even better.

Is there a standard way of doing this? What should I do about files like the makefile and the readme that live in the root folder?

If I have to start over importing to hg through some other process that shouldn't be a problem either. It just takes far too long to do standard operations with he monolithic repository, even though it already blows svn out of the water.

[Devil's advocate] Have you tried importing the whole thing into git to see how well it performs? — Marcelo Cantos, Sep 12 '11 at 22:32
Yes. I love git too. We decided hg would be better for our team because the learning curve is lower, and the windows tooling is quite nice. — captncraig, Sep 12 '11 at 23:17

score 4 · Accepted Answer · answered Sep 12 '11 at 22:36

4

You can use Mercurial's Convert Extension (which you probably already know since you converted the repository from SVN to HG) with the -filemap option to create separate HG repositories for the SVN subfolders.
You need the include directive in particular:

The include directive causes a file, or all files under a directory, to be included in the destination repository, and the exclusion of any other element that's not under an inclusion rule.

Then, you could use Mercurial Subrepositories to set up the repositories in the desired folder structure:

Subrepositories is a feature that allows you to treat a collection of repositories as a group. This will allow you to clone, commit to, push, and pull projects and their associated libraries as a group.

answered Sep 12 '11 at 22:36

Christian Specht

35,843
15
128
182

I knew it was called something like that, but I wasn't sure what I was looking for. Subrepositories look cool. – captncraig Sep 12 '11 at 23:20
So you are saying I would run convert N number of times with different include directives, and then add the appropriate hgsub files to make the subrepositories? Or can I do the convert all in one step? – captncraig Sep 12 '11 at 23:33
I'm no convert guru, but as far as I know there's no way to do it all in one step, you need to run convert N number of times with different include directives. – Christian Specht Sep 12 '11 at 23:53
Right, there's no way to do it in one step. Also, let me add that you can convert your existing Mercurial repo with the convert extension -- it does hg -> hg conversions too. – Martin Geisler Sep 14 '11 at 20:51

score 2 · Answer 2 · answered Sep 13 '11 at 03:40

My approach - which I have used successfully on smaller repositories is as follows:

Determine list smaller repositories and how they will interrelate.

clone from original repository
hg rm all files you do not want

If the subprojects are intertwined with a top-level Makefile, you can add a top-level repository which subrepositories in the smaller repositories.

This approach maintains history and does not force people to reclone. If you are okay with exploding history, convert is fine, and will shrink repository size.

A word (or a few) of warning: subrepositories have a good deal of difficult behavior as compared to the ease of one-repository operations. Some operations recurse, other's don't. Merge in particular recurses, which routinely gives me fits. Determining the entire system state can be challenging, especially as the number of subrepositories goes up (I'm helping manage a project with > 100 subrepositories, and yes, that was the logical and best way to handle it).

How to break a large mercurial repository into smaller ones

2 Answers2