One way hg to git sync

Question

I have a mercurial repository which contains a monolith project I am trying to gradually split. While doing those, I figured I would convert the new sub projects to git hence the one way sync.

A few more details about what I am doing:

• the hg repo and the new git repos are located in a private bitbucket cloud account.
• I want to keep the commits history while doing the split
• All our development is Windows based (although I'm open to do the migration using a unix based system)
• the initial repo is 7 years old, it has all sort of tags, closed branches, some branches with unsupported git characters. But more importantly I am happy if I can migrate only the default/master (if it helps me get the job done and doesn't imply losing history)
• As we are gradually converting some projects inside the repo (lets say I have 30 projects and I want incrementally to move them) I need to do one way syncs from hg to git. I am not afraid of the merges and I am happy to keep my new repo work outside of master and then just rebase with the hg changes as we go.
• I get the idea our mercurial repo is not properly configured (I saw multiple heads, etc) but I am outside my comfort zone when I dig deep into into mercurial backbone.

So far I tried several tools such as fast-export, mercurial hg hggit plugin. However I am struggling to find good step by step tools. (and almost all approaches in this thread Convert Mercurial project to Git)

fast-export was the tool that gave me the best results, I was able to migrate the project once and everything but when I tried to resync I started to get errors, like branch modified outside and multiple heads.

Now that I explained my problem in more detail I can ask the question.

What would be the best approach and tools to use for me to be able to do a one way hg to git migration?

Also, how can I make sure my mercurial repository is correctly configured to avoid any potential issues when migrating to git?

Answer 1

After countless tries, I think I found a way to do what I wanted in a consistent way. For future reference this were the steps:

Installing Necessary tools

Install Git for Windows

Install Tortoise HG or Mercurial standalone

Install Python 2.7 (fast-export does not support Python 3.X at the moment)

Open a Command Line prompt (Run as Admin).

Check if you can run git, mercurial and python as follow:

$ git

$ mercurial

$ python

If you have installed the other ones above and you are getting errors you need to set the path, in my case I only had to do it for Python. So I did:

$ setx path "%path%;C:\Python27"

restart the command prompt and everything should be ready to go.

Install fast-export and clone the mercurial and git repos

Create a clean directory so the work will be contained in there (In my case I wont use the repos inside this directory for anything other than syncing the projects). e.g:

c:\syncprojects

From inside c:\syncprojects start by cloning fast-export

$ git clone https://github.com/frej/fast-export.git fast-export

Then clone the mercurial project

$ hg clone https://bitbucket.org/user/mercurialrepo

Then clone the git project you want to sync into

$ git clone https://bitbucket.org/user/gitrepo gitrepo

It helped me to have a authors file configured correctly so I did

$ cd mercurialrepo
$ hg log | grep user: | sort | uniq | sed 's/user: *//' > ../authors

Then open the authors file which was created in c:\syncprojects make sure the authors file matches something similar to this:

bob=Bob Jones <bob@company.com>
bob@localhost=Bob Jones <bob@company.com>
bob <bob@company.com>=Bob Jones <bob@company.com>

Next step is to start the actual migration, for this step I felt more comfortable using the git bash so I did: In windows explorer, right click on the gitrepo folder and select "Git Bash here"

Then I made my local git repo case sensitive, this helps with the process but its a good thing to have, as I run in to problems with case sensitive folders in the past. Just do:

git config core.ignoreCase false 

Trigger the sync

Finally I did:

$ c:\syncprojects\fast-export\hg-fast-export.sh -r c:\syncprojects\mercurialrepo -A c:\syncprojects\authors --force

If all goes well (and this does not necessarily happen all the time, for multiple reasons I had issues with the heads in mercurial, issues with local changes in the git repo I am trying to sync into).

All we need to do is checkout the head and push the changes to remote, as such:

$ git checkout HEAD
$ git push -u origin master

The next time you want to do a sync just repeat the final part:

$ c:\syncprojects\fast-export\hg-fast-export.sh -r c:\syncprojects\mercurialrepo -A c:\syncprojects\authors --force

$ git checkout HEAD

$ git push -u origin master

Unfortunately the steps are not as fast forward as it looks but this was the more concise and consistent way I found to tackle my problem.

A few more tips:

• I did not merge any new code to master in the newly created git repo.
• Until I am totally satisfied and able to stop the sync I will have a branch that contains the changes I do in my day to day and periodically merge master back into that branch.
• Do not use this repos for development, fast-export stores data inside the repos that might get lost and make the re-syncing project very hard to achieve. Clone the repos in a separate location (and please be careful with checking other branches out in this repos for the same reason).