327

I need to merge two Git repositories into a brand new, third repository. I've found many descriptions of how to do this using a subtree merge (for example Jakub Narębski's answer on How do you merge two Git repositories?) and following those instructions mostly works, except that when I commit the subtree merge all of the files from the old repositories are recorded as new added files. I can see the commit history from the old repositories when I do git log, but if I do git log <file> it shows only one commit for that file - the subtree merge. Judging from the comments on the above answer, I'm not alone in seeing this problem but I've found no published solutions for it.

Is there any way do merge repositories and leave individual file history intact?

Community
  • 1
  • 1
Eric Lee
  • 8,181
  • 4
  • 21
  • 18
  • I'm not using Git, but in Mercurial I'd first do a convert if necessary to fix the file paths of the repos to be merged, and then force-pull one repo into the target to get the changesets, and then do a merge of the different branches. This is tested and works ;) Maybe this helps to find a solution for Git as well... compared to the subtree-merge approach I guess the convert step is different where the history is rewritten instead of just mapping a path (if I understand correctly). This then ensures a smooth merge without any special handling of file paths. – Lucero Oct 24 '12 at 00:12
  • I also found this question helpful http://stackoverflow.com/questions/1683531/how-to-import-existing-git-repository-into-another – nacross Feb 17 '14 at 01:07
  • I created a follow-up question. Might be interesting: Merge two Git repositories and keep the master history: http://stackoverflow.com/questions/42161910/merge-two-git-repositories-and-keep-the-master-history – Dimitri Dewaele Feb 27 '17 at 14:20
  • The automated solution that worked for me was https://stackoverflow.com/a/30781527/239408 – xverges Jun 01 '17 at 16:45

10 Answers10

364

It turns out that the answer is much simpler if you're simply trying to glue two repositories together and make it look like it was that way all along rather than manage an external dependency. You simply need to add remotes to your old repos, merge them to your new master, move the files and folders to a subdirectory, commit the move, and repeat for all additional repos. Submodules, subtree merges, and fancy rebases are intended to solve a slightly different problem and aren't suitable for what I was trying to do.

Here's an example Powershell script to glue two repositories together:

# Assume the current directory is where we want the new repository to be created
# Create the new repository
git init

# Before we do a merge, we have to have an initial commit, so we'll make a dummy commit
git commit --allow-empty -m "Initial dummy commit"

# Add a remote for and fetch the old repo
# (the '--fetch' (or '-f') option will make git immediately fetch commits to the local repo after adding the remote)
git remote add --fetch old_a <OldA repo URL>

# Merge the files from old_a/master into new/master
git merge old_a/master --allow-unrelated-histories

# Move the old_a repo files and folders into a subdirectory so they don't collide with the other repo coming later
mkdir old_a
dir -exclude old_a | %{git mv $_.Name old_a}

# Commit the move
git commit -m "Move old_a files into subdir"

# Do the same thing for old_b
git remote add -f old_b <OldB repo URL>
git merge old_b/master --allow-unrelated-histories
mkdir old_b
dir –exclude old_a,old_b | %{git mv $_.Name old_b}
git commit -m "Move old_b files into subdir"

Obviously you could instead merge old_b into old_a (which becomes the new combined repo) if you’d rather do that – modify the script to suit.

If you want to bring over in-progress feature branches as well, use this:

# Bring over a feature branch from one of the old repos
git checkout -b feature-in-progress
git merge -s recursive -Xsubtree=old_a old_a/feature-in-progress

That's the only non-obvious part of the process - that's not a subtree merge, but rather an argument to the normal recursive merge that tells Git that we renamed the target and that helps Git line everything up correctly.

I wrote up a slightly more detailed explanation here.

Dai
  • 141,631
  • 28
  • 261
  • 374
Eric Lee
  • 8,181
  • 4
  • 21
  • 18
  • 22
    this solution using `git mv` doesn't work so well. when you later use a `git log` on one of the moved files you only get the commit from the move. all previous history is lost. this is because `git mv` is really `git rm; git add` but [in one step](http://stackoverflow.com/a/1094392/959352). – mholm815 Feb 04 '13 at 15:34
  • 22
    It's the same as any other move/rename operation in Git: from the command line you can get all of the history by doing `git log --follow`, or all of the GUI tools do that for you automatically. With a subtree merge you **can't** get the history for individual files, as far as I know, so this method is better. – Eric Lee Feb 08 '13 at 23:47
  • 5
    @EricLee When the old_b repo is merged I get a lot of merge conflicts. Is that expected? I get CONFLICT (rename/delete) – Jon Jan 27 '14 at 10:15
  • @Jon No, if you've moved the first repo into a subdirectory first then the second repo shouldn't cause any conflicts. If it does, you've probably skipped a step somewhere or have discovered some behavior I don't know about. – Eric Lee Jan 28 '14 at 17:59
  • I'm on Git 1.8.5 on Windows and it always causes issues when running the merge on the old_b and i'm not sure why. CONFLICT (rename/delete): DeleteFeedCommand.cs deleted in HEAD and renamed in domain/master. Version domain/master of DeleteFeedCommand.cs left in tree. – Jon Jan 29 '14 at 13:04
  • domain/master is old_b here – Jon Jan 29 '14 at 13:06
  • 1
    I think git add -u on the conflict seems to get around it – Jon Jan 29 '14 at 15:26
  • I was merging B repo into A repo. I had a feature-in-progress branch in both repo. I wanted to pull in-progress changes of B repo to A repo. In A repo, I did `git checkout feature-in-progress` > `merge master` > `git merge -s recursive -Xsubtree=B old_a/feature-in-progress`. It merged properly but the newly added files in `B/feature-in-progress` are missing. :( – IsmailS Jul 30 '15 at 14:33
  • 14
    When I attempt "dir -exclude old_a | %{git mv $_.Name old_a}", I get sh.exe": dir: command not found and sh.exe": git: command not found. Using this works: ls -I old_a | xargs -I '{}' git mv '{}' old_a/ – George Nov 11 '15 at 02:40
  • I am using Git Bash on Windows and had the same problem as George. His command works, but it's not clear that the switches to both 'ls' and 'xargs' is a capital 'eye' and not a lower 'el'. – Mageician Nov 24 '15 at 15:47
  • 7
    This is `1` (the number One) for `ls`and capital 'eye' for `xargs`. Thanks you for this tip! – Dominique Vial Jan 26 '16 at 11:14
  • This method of combining git repositories together should be carefully evaluated if you are planning on using code from new git repository for build/release process. Looks like git move operation is messing up all branches and tags on the code in newly created repository. Although you can see your tags in place but it's not usable at all. – user3356885 Feb 23 '16 at 18:15
  • I am evaluating this option and it seems to be working quite well. One issue that I am running into is that the git logs for some files contain _many_ more commits after merging. A file that had 4 commits in it's original repo has 400 after merging into a repo with 4000 combined commits. I'm not sure where it is picking up the other commits. Any ideas? – Jeffery Utter Apr 13 '16 at 03:00
  • 1
    +1, but note that if you have a .gitignore in both the source repos then you will get a conflict on the second merge, which must be fixed. Note also that someone else edited your answer to add `--allow-unrelated-histories` to your merge command. This is new, and isn't in your blog, and should probably be reverted... – EML Feb 09 '17 at 14:25
  • Had to do this and your steps above worked great, thanks! Just wondering if it is safe to remove the references to the old remotes at the end and indeed whether the old repositories can be nuked, or whether the history in the merged repo still points at the old repositories? – Matthew Wise Feb 23 '17 at 11:35
  • https://stackoverflow.com/a/30781527/239408 provides another automated approach, based on bash – xverges Jun 01 '17 at 16:44
  • 4
    On osx/macos the ```dir -exclude old_a | %{git mv $_.Name old_a}``` could be ```ls | grep -v 'old_a' | xargs -I '{}' git mv '{}' old_a``` – hussfelt Sep 13 '17 at 11:24
  • doesn't always work. when merging multiple projects that have the same directory layout, there will be a massive amount of conflicts due to the merge attempt of the next subproject following a `git mv`. – axd Oct 20 '17 at 17:26
  • 1
    In a bash context I prefer: `find . -maxdepth 1 -not -name ".git" -and -not -name 'old_a' -exec git mv {} old_a/ \;` – acumartini Mar 20 '18 at 23:07
  • `dir --ignore old_a` not `-e` – Jeremy Dec 05 '18 at 16:46
  • As mholm815 said, after merging the repos simply using `git log ` will stop on the moving commit. Eric Lee suggested using `git log --follow ` but it didn't work for me (git 2.25). Instead, it showed no commits at all. To get full history for a particular file I use `git log -m --follow `. – Peter Feb 09 '20 at 01:32
  • 1
    To make the move easier, I would just `mkdir -p old_a && git mv -k * old_a`. Much simpler. – Robert Massaioli Sep 04 '21 at 02:06
  • Following the comment of @RobertMassaioli, if you need to move hidden files/folders as well, you can do `mkdir -p old_a && git mv -k .[!.]* old_a` – Astariul Mar 07 '22 at 06:37
  • Do I have to execute this in PowerShell or in GitBash? – erik-stengel Mar 30 '22 at 03:40
  • I completed the suggestions above for Unix shell/bash to consistently consider hidden files but still exclude ., .. and .git, and also work with multiple subfolder excludes. Note that you can `git mv -k` to avoid excluding `.` and the move target subdir. Assuming you have already added old_a and are now merging old_b: 1. With ls+grep+xargs `ls -A | grep -vw '.git\|old_a\|old_b' | xargs -I '{}' git mv '{}' old_b` 2. With find+exec: `find . -maxdepth 1 -not -name "." -not -name ".git" -and -not -name 'old_a' -and -not -name 'old_b' -exec git mv '{}' old_b \;` Don't hesitate to dry run with -n! – hsandt May 24 '23 at 12:52
201

Here's a way that doesn't rewrite any history, so all commit IDs will remain valid. The end-result is that the second repo's files will end up in a subdirectory.

  1. Add the second repo as a remote:

    cd firstgitrepo/
    git remote add secondrepo username@servername:andsoon
    
  2. Make sure that you've downloaded all of the secondrepo's commits:

    git fetch secondrepo
    
  3. Create a local branch from the second repo's branch:

    git branch branchfromsecondrepo secondrepo/master
    
  4. Move all its files into a subdirectory:

    git checkout branchfromsecondrepo
    mkdir subdir/
    git ls-tree -z --name-only HEAD | xargs -0 -I {} git mv {} subdir/
    git commit -m "Moved files to subdir/"
    
  5. Merge the second branch into the first repo's master branch:

    git checkout master
    git merge --allow-unrelated-histories branchfromsecondrepo
    

Your repository will have more than one root commit, but that shouldn't pose a problem.

Flimm
  • 136,138
  • 45
  • 251
  • 267
  • 1
    Step 2 doesn't work for me: fatal: Not a valid object name: 'secondrepo/master'. – Keith Jan 31 '14 at 15:47
  • @Keith: make sure you've added the second repo as a remote named "secondrepo", and that that repo has a branch named "master" (you can view branches on a remote repo with the command `git remote show secondrepo`) – Flimm Feb 04 '14 at 10:59
  • I had to do a fetch to bring it down as well. In between 1 and 2 I did git fetch secondrepo – sksamuel Mar 15 '14 at 15:27
  • @monkjack: I've edited my answer to include a git fetch step. Feel free to edit the answer yourself in future. – Flimm Mar 17 '14 at 11:26
  • This isn't working, I think one step is missing or wrong, because I'm trying to merge like this tutorial, and all I got is just Unmerged error in many files, besides all my main files go into the sub directory folder.... – DarkteK Apr 13 '15 at 20:51
  • This answer worked for me (or more specifically, for the colleague who just asked me how to do this). He didn't need to move any files to a subdirectory, since the two repos had different directory structures in them (C# project, merging the FooProject and BarProject repositories together -- everything was already under folders named FooProject or BarProject). Had a couple of easily-solved merge conflicts on files like `.gitignore`, but that's it. All FooProject and BarProject files are now in one repo, with full history preserved for each. – rmunn Mar 21 '16 at 09:26
  • For part 5, perhaps you like to use `git merge branchfromsecondrepo -s recursive -X no-renames` so git don't tries to find renamed files. – Fredrik Erlandsson Apr 28 '16 at 07:12
  • After Step 4 and before Step 5, are you supposed to push `branchfromsecondrepo` back to its corresponding remote? Because when I went to step 5, switched back the the `master` branch, and then tried to `git merge branchfromsecondrepo/master`, the changes I made in Step 4 were no longer applied, it was as if it merged straight from the remote and not from the local `branchfromsecondrepo`. I was unable to run `git merge branchfromsecondrepo`, git gave an error about it not being something that could be merged – user5359531 Jul 08 '16 at 10:35
  • @user5359531 Nope, there's no need to push anything after step 4, no changes are made to the remote at all. If you like the result, you can push the result after all the steps in my answer have been completed. In Step 5, you should merge with `branchfromsecondrepo`, and not `secondrepo/master` or `branchfromsecondrepo/master` or any remote-tracking branch. I think you could might be confused about the differences between local branches, remote-tracking branches and branches on remotes, see this post: http://stackoverflow.com/a/24785777/247696 – Flimm Jul 08 '16 at 12:54
  • Step 5 requires `--allow-unrelated-histories`. – Mikhail Orlov Aug 30 '16 at 09:03
  • Sorry @Xiong Chiamiov, I rolled back your edit because I prefer splitting a command like `git checkout -b BLA BLA2` into two commands when I'm teaching someone something. Also, your edit made the command no longer match the description of the command before it. – Flimm Aug 31 '16 at 09:11
  • Once I've done all of those steps, can I safely delete the branch branchfromsecondrepo? Or will doing that result in loss of history? – Eric Smith Dec 02 '16 at 10:35
  • @EricSmith You can safely delete it. `git branch -d branchfromsecondrepo` won't delete the branch unless it's safe to do so. (The `-D` flag forces a deletion, but `-d` is safe.) – Flimm Dec 02 '16 at 10:53
  • Note that the `--allow-unrelated-histories` option to git merge in step 5 is relatively new. Git 2.7 (the default version in Ubuntu 16.04) doesn't know it so I installed Git 2.13 from the git-core PPA. The procedure works great! – Martijn Heemels Aug 10 '17 at 14:45
  • 4
    @MartijnHeemels For older version of Git, just omit `--allow-unrelated-histories`. See the history of this answer post. – Flimm Aug 10 '17 at 14:46
  • Great answer, helped me out. :) . For the last step, what is the reason to do a `merge` rather than a `rebase`? I did `git rebase branchfromsecondrepo` to have the 1st repo start off the history of the second repo and I would imagine that's more preferable in this type of situation. – C S May 07 '18 at 01:35
  • Step 4 does not work in Windows (in Powershell), xargs command needs to be substituted. Anyone have an idea how to rewrite Step 4 for Windows? @Flimm? – Lucademicus Jan 09 '19 at 05:20
  • @Lucademicus I saw your suggested edit. Initially I approved it, but then I rolled back, as it is not strictly equivalent to the bash equivalent. I'm not sure if it handles Git ignored files correctly. – Flimm Jan 09 '19 at 14:04
  • This worked great, except I'm missing the submodules now – Tim MB Nov 12 '21 at 17:45
  • This solution works for me. All the previous commit messages from the other repo are kept!! – Sta_Doc Jan 12 '22 at 14:50
  • Note: step 4 can be done with [git filter-repo](https://github.com/newren/git-filter-repo) in a more concise, cross platform manner: `git filter-repo --refs branchfromsecondrepo --path src --path ... --to-subdirectory-filter subdir/`. You must specify a `--path` for every file/folder you wish to copy. This will make it appear as though the files were there all along. – Ryan Wheale Jan 06 '23 at 21:44
  • @RyanWheale `git filter-repo` edits the history of the repo, which step 4 does not do. They are not equivalent. OP asked how to "merge two Git repositories without breaking file history". – Flimm Jan 10 '23 at 08:23
  • @Flimm - It only rewrites the history so far as to make it appear that the files were located inside `subdir/` all along. It preserves all other aspects of the history (eg. blame still works). I am currently using this to merge a dozen or more repos together. The main point of my comment was to convey there's a cross platform alternative which makes the files appear to have always been there... whereas the above code (`git mv`) tracks the file as being moved from one location into the `subdir` - which may or may not be desirable. – Ryan Wheale Jan 10 '23 at 19:22
  • Note that nowadays the `master` branch is often called `main`, and you'll might have to change the branch name in the instructions. – Simon Baars Jan 17 '23 at 23:01
  • @RyanWheale I think the default assumption should be to keep things as they are, or else a merge wouldn’t be used in the first place. There is by default no need to fudge with things in order to make it appear that the files were “always there”—the history tells you where they came from with `git log --follow`. – Guildenstern Apr 16 '23 at 11:40
39

Say you want to merge repository a into b (I'm assuming they're located alongside one another):

cd b
git remote add a ../a
git fetch a
git merge --allow-unrelated-histories a/master
git remote remove a

In case you want to put a into a subdirectory do the following before the commands above:

cd a
git filter-repo --to-subdirectory-filter a
cd ..

For this you need git-filter-repo installed (filter-branch is discouraged).

An example of merging 2 big repositories, putting one of them into a subdirectory: https://gist.github.com/x-yuri/9890ab1079cf4357d6f269d073fd9731

More on it here.

x-yuri
  • 16,722
  • 15
  • 114
  • 161
  • Is it possible to do it without merge conflicts? – Bob Dec 17 '20 at 16:50
  • @Mikhail Yes, it is possible, do you see merge conflicts in the gist? If you run into merge conflicts, that means you have e.g. file `a/b/c` in both repositories. Either rename files before merge, or merge into a subdirectory, or resolve the conflicts. – x-yuri Dec 17 '20 at 18:18
  • ok. thank you. resolve the conflicts it is – Bob Dec 17 '20 at 18:31
  • 1
    this is the perfect solution for preserving file histories without becoming dependent on `--follow`, thank you! – Ishmaeel Dec 15 '21 at 10:42
  • 1
    Note that `git filter-repo` will rewrite the history. You can use `git mv` if you don’t want to do that. – Guildenstern Apr 16 '23 at 10:37
  • Upvoted. Thank you. I wrote a [much more extensive example of this, with a lot more detail, here](https://stackoverflow.com/a/76831513/4561887). – Gabriel Staples Aug 03 '23 at 20:48
33

A few years have passed and there are well-based up-voted solutions but I want to share mine because it was a bit different because I wanted to merge 2 remote repositories into a new one without deleting the history from the previous repositories.

  1. Create a new repository in Github.

    enter image description here

  2. Download the newly created repo and add the old remote repository.

    git clone https://github.com/alexbr9007/Test.git
    cd Test
    git remote add OldRepo https://github.com/alexbr9007/Django-React.git
    git remote -v
    
  3. Fetch for all the files from the old repo so a new branch gets created.

    git fetch OldRepo
    git branch -a
    

    enter image description here

  4. In the master branch, do a merge to combine the old repo with the newly created one.

    git merge remotes/OldRepo/master --allow-unrelated-histories
    

    enter image description here

  5. Create a new folder to store all the new created content that was added from the OldRepo and move its files into this new folder.

  6. Lastly, you can upload the files from the combined repos and safely delete the OldRepo from GitHub.

Hope this can be useful for anyone dealing with merging remote repositories.

abautista
  • 2,410
  • 5
  • 41
  • 72
  • 9
    This is the only solution that worked for me to preserve git history. Don't forget to remove the remote link to old repo with `git remote rm OldRepo`. – Célia Doolaeghe Jan 23 '20 at 13:45
  • 4
    I can't upvote this enough. A perfectly simple, successful, sensible solution. Thank you! And thank you @Harubiyori for the final touch. – code4meow Aug 11 '20 at 23:06
  • 1
    I found this solution straightforward to follow and it gets my vote. Thanks @abautista – Donnacha Oct 13 '22 at 13:07
  • These are the same steps as documented in the [accepted answer](https://stackoverflow.com/a/14470212/1725151) except that you only move one of the repos into a subdirectory (instead of both of them). – Guildenstern Apr 16 '23 at 11:46
8

please have a look at using

git rebase --root --preserve-merges --onto

to link two histories early on in their lives.

If you have paths that overlap, fix them up with

git filter-branch --index-filter

when you use log, ensure you "find copies harder" with

git log -CC

that way you will find any movements of files in the path.

Adam Dymitruk
  • 124,556
  • 26
  • 146
  • 141
7

I turned the solution from @Flimm this into a git alias like this (added to my ~/.gitconfig):

[alias]
 mergeRepo = "!mergeRepo() { \
  [ $# -ne 3 ] && echo \"Three parameters required, <remote URI> <new branch> <new dir>\" && exit 1; \
  git remote add newRepo $1; \
  git fetch newRepo; \
  git branch \"$2\" newRepo/master; \
  git checkout \"$2\"; \
  mkdir -vp \"${GIT_PREFIX}$3\"; \
  git ls-tree -z --name-only HEAD | xargs -0 -I {} git mv {} \"${GIT_PREFIX}$3\"/; \
  git commit -m \"Moved files to '${GIT_PREFIX}$3'\"; \
  git checkout master; git merge --allow-unrelated-histories --no-edit -s recursive -X no-renames \"$2\"; \
  git branch -D \"$2\"; git remote remove newRepo; \
}; \
mergeRepo"
Community
  • 1
  • 1
Fredrik Erlandsson
  • 1,279
  • 1
  • 13
  • 22
4

This function will clone remote repo into local repo dir:

function git-add-repo
{
    repo="$1"
    dir="$(echo "$2" | sed 's/\/$//')"
    path="$(pwd)"

    tmp="$(mktemp -d)"
    remote="$(echo "$tmp" | sed 's/\///g'| sed 's/\./_/g')"

    git clone "$repo" "$tmp"
    cd "$tmp"

    git filter-branch --index-filter '
        git ls-files -s |
        sed "s,\t,&'"$dir"'/," |
        GIT_INDEX_FILE="$GIT_INDEX_FILE.new" git update-index --index-info &&
        mv "$GIT_INDEX_FILE.new" "$GIT_INDEX_FILE"
    ' HEAD

    cd "$path"
    git remote add -f "$remote" "file://$tmp/.git"
    git pull "$remote/master"
    git merge --allow-unrelated-histories -m "Merge repo $repo into master" --edit "$remote/master"
    git remote remove "$remote"
    rm -rf "$tmp"
}

How to use:

cd current/package
git-add-repo https://github.com/example/example dir/to/save

Notice. This script can rewrite commits but will save all authors and dates, it means new commits will have another hashes, and if you try to push changes to remote server it can be able only with force key, also it will rewrite commits on server. So please make backups before to launch.

Profit!

Andrey Izman
  • 1,807
  • 1
  • 24
  • 27
  • I'm using zsh rather than bash, and v2.13.0 of git. No matter what I've tried, I haven't been able to get `git filter-branch --index-filter` to work. Typically I get an error message that the .new index file doesn't exist. Does that ring any bells? – Patrick Beard Jul 11 '17 at 22:52
  • @PatrickBeard I don't know zsh, you can create separated file `git-add-repo.sh` with function above, at the end of the file put this line `git-add-repo "$@"`. After that you can use it from zsh like `cd current/git/package` and `bash path/to/git-add-repo.sh https://github.com/example/example dir/to/save` – Andrey Izman Jul 12 '17 at 00:22
  • The problem was discussed here: https://stackoverflow.com/questions/7798142/error-combining-git-repositories-into-subdirs `mv "$GIT_INDEX_FILE.new" "$GIT_INDEX_FILE"` fails sometimes, so you have to add an `if test`. – Patrick Beard Jul 12 '17 at 02:20
  • 1
    I would not use this method! I tried the script, naively and verbatim (I can only blame myself for that part), and it clobbered my local git repo. The history looked mostly right, but doing a git push back to Github resulted in the dreaded "RPC failed; curl 55 SSL_write() returned SYSCALL, errno = 32" error. I tried to repair it, but it was irreparably broken. I ended up having to reconstruct things in a new local repo. – Mason Freed Feb 01 '20 at 17:30
  • @MasonFreed this script creates a new git history with mix of both repos, so it can't be pushed to old repo, it require to create a new one or push with force key, means it rewrite your repo on server – Andrey Izman Feb 02 '20 at 21:09
2

Follow the steps to embed one repo into another repo, having one single git history by merging both git histories.

  1. Clone both the repos you want to merge.

git clone git@github.com:user/parent-repo.git

git clone git@github.com:user/child-repo.git

  1. Go to child repo

cd child-repo/

  1. run the below command, replace path my/new/subdir (3 occurences) with directory structure where you want to have the child repo.

git filter-branch --prune-empty --tree-filter ' if [ ! -e my/new/subdir ]; then mkdir -p my/new/subdir git ls-tree --name-only $GIT_COMMIT | xargs -I files mv files my/new/subdir fi'

  1. Go to parent repo

cd ../parent-repo/

  1. Add a remote to parent repo, pointing path to child repo

git remote add child-remote ../child-repo/

  1. Fetch the child repo

git fetch child-remote

  1. Merge the histories

git merge --allow-unrelated-histories child-remote/master

If you check the git log in the parent repo now, it should have the child repo commits merged. You can also see the tag indicating from the commit source.

Below article helped me in Embedding one repo into another repo, having one single git history by merging both git histories.

http://ericlathrop.com/2014/01/combining-git-repositories/

Hope this helps. Happy Coding!

AnoopGoudar
  • 914
  • 9
  • 18
  • Step 3 failed for me with syntax error. Semi-colons are missing. Fix `git filter-branch --prune-empty --tree-filter ' if [ ! -e my/new/subdir ]; then mkdir -p my/new/subdir; git ls-tree --name-only $GIT_COMMIT | xargs -I files mv files my/new/subdir; fi'` – Yuri L Feb 24 '20 at 05:46
1

I created a Repository with some scripts based on the answer from x-yuri which uses filter-repo. With my scripts you are able to easily move all branches and tags into your new repository without getting merge conflicts if you specify different subdirs.

Devpool
  • 701
  • 7
  • 6
0

How to merge 1 or 2 or more local-only (or remote, but cloned locally) repos into another repo, while retaining git history

I need something a little different from other answers here, and without installing any new tools, and with more details to understand what's happening here better, so here's my answer:

Let's assume you have this local file structure. Both repos here may be stored locally only, or also on a remote URL. The instructions are the same either way, so long as they are stored locally also, meaning that if they are remote repos, you've already cloned them locally.

You currently have this directory structure:

repo1 and repo2 here are independent, stand-alone, locally-stored or cloned git repos.

repo1/
    .git/
    .gitignore
    (other files and folders)

repo2/
    .git/
    .gitignore
    (other files and folders)

What you want is to end up with this:

repo1 and repo2 here are just folders (not repos nor subrepos/submodules) inside new_repo.

repo1/      # will be deleted when done
    ...

repo2/      # will be deleted when done
    ...

new_repo/
    .git/
    .gitignore
    repo1/
        (other files and folders)
    repo2/
        (other files and folders)

Detailed Instructions

  1. Prepare the old repos. This can be done with any number of repos you are combining, whether it be 1, 2, or 100 repos. In this example I will just do two repos, repo1 and repo2. To prepare them for a merge into a single, outer repo, while retaining their history, we need to first put their contents into a subdir by the same name as their repo names.

    So, we will go from this:

    repo1/
        .git/
        .gitignore
        (other files and folders)
    
    repo2/
        .git/
        .gitignore
        (other files and folders)
    

    to this:

    repo1/
        .git/
        repo1/
            .gitignore
            (other files and folders)
    
    repo2/
        .git/
        repo2/
            .gitignore
            (other files and folders)
    

    Commands to run to do this:

    # 1. Fix up repo1
    
    cd path/to/repo1
    mkdir repo1
    # move all non-hidden files and folders into `repo1/`
    mv * repo1/
    # move all hidden files and folders into `repo1/`
    mv .* repo1/
    # Now move the .git dir back to where it belongs, since it was moved by the
    # command just above
    mv repo1/.git .
    # commit all these changes into this repo
    git add -A
    git status
    git commit -m "Move all files & folders into a subdir"
    
    
    # 2. Fix up repo2 (same process as just above, except use `repo2` instead of
    # `repo1`)
    
    cd path/to/repo2
    mkdir repo2
    mv * repo2/
    mv .* repo2/
    mv repo2/.git .
    git add -A
    git status
    git commit -m "Move all files & folders into a subdir"
    
  2. Create the new_repo, if necessary. If this repo already exists, that's fine, use it as-is, and skip this step. If you need to create this as a brand-new repo, here's how:

    # Create `new_repo`
    
    cd path/to/parentdir_of_repo1_and_repo2
    mkdir new_repo
    cd new_repo
    
    git init
    # (Optional, but recommended) rename the main branch from `master` to `main`
    git branch -m main
    # set your name and email for just this repo if you haven't done this
    # globally previously
    git config user.name "First Last"
    git config user.email firstlast@gmail.com
    
    # create an empty first commit to start a git history
    git commit --allow-empty -m "Initial empty commit"
    
  3. Merge our fixed-up repo1 and repo2 repo histories and contents into new_repo.

    cd path/to/new_repo
    
    # --------------------------------------------------------------------------
    # 1. Merge repo1, with all files and folders and git history, into new_repo
    # --------------------------------------------------------------------------
    
    # Add repo1 as a local "remote" named `repo1`
    # - Note: this assumes that new_repo, repo1, and repo2 are all at the same
    #   directory level and inside the same parent folder. If this is *not* the
    #   case, no problem. Simply change `"../repo1"` below to the proper
    #   relative *or* absolute path to that repo! Ex: `"path/to/repo1"`. 
    git remote add repo1 "../repo1"
    # View all of your remotes. 
    # - You'll now see `repo1` as a remote which points to the local "URL"
    #   of "../repo1"
    git remote -v
    
    # Fetch all of repo1's files and history into new_repo's .git dir. 
    # - Note that `repo1` here is the name of the remote alias that you just
    #   added above.
    git fetch repo1 
    # View the new locally-stored, remote-tracking hidden branch that was just
    # created for you. 
    # - `git branch -r` will now show a new hidden branch named `repo1/main` or
    #   `repo1/master` or whatever your main branch was named there.
    git branch -r
    
    # Assuming that your new hidden branch that you just fetched is called
    # `repo1/main`, let's merge that into our currently-checked-out branch now.
    # - This merges repo1's files and folders and git history into new_repo.
    # - change `repo1/main` to `repo1/some_other_branch` if you want to merge in
    #   `some_other_branch` instead.
    # - Immediately after running this command, you will now see a new folder,
    #   `repo1`, with all of its files and folders within it, created inside
    #   the `new_repo` directory.
    git merge --allow-unrelated-histories repo1/main
    # Since you have independent sub-folders prepared inside repo1 and repo2,
    # there will be no conflicts whatsoever. When the git merge editor opens
    # up, just save and close it to complete the merge. Optionally, add any
    # comments you wish before saving and closing it.
    
    # Now, remove the remote named "repo1", since we no longer need it.
    git remote remove repo1
    # View all remotes to ensure it is now gone
    git remote -v
    
    # See the new commit status. If `repo1` had 100 commits in it, for instance,
    # `git status` will now show this, since `new_repo` now has those 100
    # commits plus this new merge commit in it:
    #       
    #       $ git status
    #       On branch main
    #       Your branch is ahead of 'origin/main' by 101 commits.
    #         (use "git push" to publish your local commits)
    #       
    #       nothing to commit, working tree clean
    #       
    git status 
    # Push to a remote, if you have one configured
    git push
    
    # You may now optionally manually delete the original `repo1` since it is
    # merged into `new_repo`. Here's what that command would look like:
    # - WARNING: get this path correct or you'll delete a whole lot of stuff you
    #   don't mean to!
    # 
    #       rm -r ../repo1
    #
    
    # --------------------------------------------------------------------------
    # 2. Merge repo2 in too. 
    # - This is the exact same process as above, except we use `repo2` instead
    #   of `repo1`.
    # --------------------------------------------------------------------------
    
    git remote add repo2 "../repo2"
    git remote -v
    
    git fetch repo2 
    git branch -r
    
    git merge --allow-unrelated-histories repo2/main
    
    git remote remove repo2
    git remote -v
    
    git status 
    git push
    # Optional: `rm -r ../repo2`
    
  4. (Optional) view your new, merged history inside new_repo with git lg:

    cd path/to/new_repo
    
    # Add this really great `git lg` alias from here:
    # https://coderwall.com/p/euwpig/a-better-git-log
    git config --global alias.lg "log --color --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%cr) %C(bold blue)<%an>%Creset' --abbrev-commit"
    
    # now view the new git history; it will cleanly show you the branch pattern,
    # and your new contents, including their branch lines
    git lg
    

    If you run git lg after merging in repo1 only, here's what you might see, assuming that repo1 had only 7 commits in it:

    $ git lg
    *   32c1d05 - (HEAD -> main, origin/main, origin/HEAD) Merge remote-tracking branch 'repo1/main' (x minutes ago) <Gabriel Staples>
    |\  
    | * eca4e6c - Move all files & folders into a subdir (x minutes ago) <Gabriel Staples>
    | * 92853ac - my repo1 commit message <Gabriel Staples>
    | * 07fe88e - my repo1 commit message <Gabriel Staples>
    | * f3b0847 - my repo1 commit message <Gabriel Staples>
    | * bcb0ca5 - my repo1 commit message <Gabriel Staples>
    | * 712ec94 - my repo1 commit message <Gabriel Staples>
    | * 1103fe1 - Initial commit (x days ago) <Gabriel Staples>
    * 47a663b - my new_repo commit message <Gabriel Staples>
    * 5d865d0 - my new_repo commit message <Gabriel Staples>
    * d47c4e1 - Initial commit (x hours ago) <Gabriel Staples>
    

References

  1. The bulk of my instructions were learned here: https://blog.jdriven.com/2021/04/how-to-merge-multiple-git-repositories/
  2. this very similar answer, by @x-yuri
  3. Super User: How to copy with cp to include hidden files and hidden directories and their contents?

See also

In addition to the references above, see:

  1. https://gfscott.com/blog/merge-git-repos-and-keep-commit-history
  2. the main answer, by @Eric Lee
Gabriel Staples
  • 36,492
  • 15
  • 194
  • 265