git move directory to another repository while keeping the history

Question

First - apologies for asking this question. There are a lot of topics about it already. But I'm not having much luck with them. My lack of familiarity with git is not much help.

I'm moving a folder from one git repo to another (which already exists). e.g.

repo-1
---- dir1
---- dir2
---- dir3
---- dir-to-move
---- dir5

repo-2
---- dir1
---- dir2
---- dir3

In the end I want the repos to look like

repo-1
---- dir1
---- dir2
---- dir3
---- dir-to-move
---- dir5

repo-2
---- dir1
---- dir2
---- dir3
---- dir-to-move

i.e. For a time dir-to-move will exist in both repos. But eventually I'll migrate the latest changes to repo-2 and remove dir-to-move from repo-1.

My initial research made me believe that I needed to use filter-branch. e.g.

How to move files from one git repo to another preserving history using `git format-patch`and `git am`

I've since learnt that subtree superseded that approach. However it's not doing what I expected. I thought I'd be able to do something like

In repo-1 workspace

git subtree split -P dir-to-move -b split

to filter the split branch down to only dir-to-move and it's history. Then in repo-2 workspace

git remote add repo-1 repo-1-url.git
git subtree add --prefix dir-to-move split

This does move the code across. It also, sort of, includes the history

e.g.

cd repo-2
git log

Shows commits from repo-1

but

cd repo-2
git log dir-to-move

Shows only an 'Add dir-to-move from commit ....'

i.e. The history is included but does not show up when checked for the specific files/directories.

How can I do this properly?

Answer 1

This is indeed possible using git subtree.

In repo-1 create a subtree:

git subtree split -P dir-to-move -b <split>

The split branch will now only contain the dir-to-move directory. You now need to pull the branch from repo-1 into a branch in repo-2.

In case repo-2 happens to be a new repository (e.g. just after git init) Things are as easy as checking out a branch with no history and pulling from repo-1.

cd repo-2
git checkout <branch>
git pull <path-to-repo-1.git> <split>

However if repo-2 is an existing repo where commits have already been made (as is the case in this question), you need to merge from an orphan branch in repo-2:

cd repo-2
git checkout --orphan <temp>                    # Create a branch with no history
git pull <path-to-repo-1.git> <split>           # Pull the commits from the subtree
git checkout <branch>                           # Go back to the original branch
git merge --allow-unrelated-histories <temp>    # Merge the unrelated commits back
git branch -d <temp>                            # Delete the temporary branch

Answer 2

I can't help you with git subtree, but with filter-branch it's possible.

First you need to create a common repository that will contain both source and destination branches. This can be done by adding a new "remote" beside "origin" and fetching from the new remote.

Use filter-branch on the source branch to rm -rf all directories except dir-to-move. After that you'll have a commit history that can be cleanly rebased or merged into the destination branch. I think the easiest way is to cherry-pick all non-empty commits from the source branch. The list of these commits can be obtained by running git rev-list --reverse source-branch -- dir-to-move

Of course, if the history of dir-to-move is non-linear (already contains merge commits), then it won't be preserved by cherry-pick, so git merge can be used instead.

Example create common repo:

cd repo-2
git remote add source ../repo-1
git fetch source

Example filter branch

cd repo-2
git checkout -b source-master source/master
CMD="rm -rf dir1 dir2 dir3 dir5"
git filter-branch --tree-filter "$CMD"

Example cherry-pick into destination master

cd repo-2
git checkout master
git cherry-pick `git rev-list --reverse source-master -- dir-to-move`

Answer 3

FWIW, the following worked for me after a number of iterations.

Clone both the repos to a temporary work area.

git clone <repourl>/repo-1.git 
git clone <repourl>/repo-2.git
cd repo-1
git remote rm origin # delete link to original repository to avoid any accidental remote changes
git filter-branch --subdirectory-filter dir-to-move -- --all  # dir-to-move is being moved to another repo.  This command goes through history and files, removing anything that is not in the folder.  The content of this folder will be moved to root of the repo as a result. 
# This folder has to be moved to another folder in the target repo.  So, move everything to another folder.
git filter-branch -f --index-filter \
'git ls-files -s | /usr/local/bin/sed -e "s/\t\"*/&dir-to-move\//" |
    GIT_INDEX_FILE=$GIT_INDEX_FILE.new \
        git update-index --index-info &&
 mv "$GIT_INDEX_FILE.new" "$GIT_INDEX_FILE"' HEAD
# Above command will go through history and rewrite the history by adding dir-to-move/ to each files.  As a result, all files will be moved to a subfolder /dir-to-move.  Make sure to use gnu-sed as OSX sed doesn't handle some extensions correctly.  For eg. \t

Now switch to target repo and fetch everything from source.

git clone repo-2
git remote add master ../repo-1/
git pull master master --allow-unrelated-histories
git push  # push everything to remote 

Above steps assume that master branches are used for both source and targets. However, tags and branches were ignored.

Answer 4

My approach to move dir-to-move to repo 2. Start by cloning the repo 1 in a new location and cd to the repo 1 folder. In my case I am moving the folder from repo 1 branch develop to repo 2 where develop is not existent yet.

git filter-branch --subdirectory-filter dir-to-move -- --all     #line 1
git remote -v                                                    #line 2
git remote set-url origin repo_2_remote_url                      #line 3
git remote -v                                                    #line 4
git push origin develop                                          #line 5

Explanation for each line:

1. Cleans all commits not related to dir-to-move, removes all files outside that folder, moves all contents in the root folder repo 1. From git docs:

Only look at the history which touches the given subdirectory. The result will contain that directory (and only that) as its project root

2. You are still pointing to your origin repo 1 url
3. Replace the origin url for your current folder to point to repo 2
4. Check the URLs have been updated correctly
5. Push to repo 2 the (newly created, in this case) develop branch

You can now clone or pull or fetch your repo 2 repository. You will have under repo 2 folder the contents of dir-to-move along with relevant history as expected.

Answer 5

Another solution is to use git-filter-repo which git filter-branch officially recommends.

Find a full example including hints about the installation for windows users at my answer on how to push commits affecting a given path to a new origin?

Answer 6

My favorite solution, which I've been using for years, is http://blog.neutrino.es/2012/git-copy-a-file-or-directory-from-another-repository-preserving-history/ . However, it seems to be falling off google search, so I worry it might vanish some day, so I'm reproducing it here:

mkdir /tmp/mergepatchs
cd ~/repo/org
export reposrc=myfile.c #or mydir
git format-patch -o /tmp/mergepatchs $(git log $reposrc|grep ^commit|tail -1|awk '{print $2}')^..HEAD $reposrc
cd ~/repo/dest
git am /tmp/mergepatchs/*.patch

Answer 7

git: Move folder from one repository to another while keeping history

cd target-repo
git remote add source ../source-repo
git fetch source
git checkout -b source source/master

# filter out history of a single folder
git filter-branch --subdirectory-filter ./dir -- --all

# put the folder history on top of your target repo history
git rebase master

Answer 8

We have two approaches here:

• git filter-branch
• git subtree

I used the former approach from https://www.johno.com/move-directory-between-repos-with-git-history and I found some better simplifications (e.g. Basin's approach is great).

Nevertheless, the later approach is much better and I can point some reasons:

1. "git filter-branch has a plethora of pitfalls that can produce non-obvious manglings of the intended history rewrite (and can leave you with little time to investigate such problems since it has such abysmal performance). These safety and performance issues cannot be backward compatibly fixed and as such, its use is not recommended." - You can read more about the reason why in https://git-scm.com/docs/git-filter-branch

2. I really disliked the ideia of using git filter-branch when I saw it rewrote the whole history of my repo unnecessarily (i.e. git filter-branch --subdirectory-filter my-dir -- -- all)

3. When I found git subtree split -P dir-to-move -b which creates a clean branch with only my folder I loved it, so using the approach presented by Thymo really was so clean and I strongly recommend using this approach!

Maybe a need to post a script on githug.

Answer 9

You can even make a commit on repo-1 by deleting all directory other than dir-to-move locally (by this way no need to use fitler-branch) and then pull to repo-2.

git clone https://path/to/repo-1
cd repo-1
rm dir1 dir2 dir3 dir5 -rf
git commit -m "Removed unwanted directory for repo-2
cd ../
git clone https://path/to/repo-2
cd repo-2
git remote add repo-1-src ../repo-1
git pull repo-1-src master --allow-unrelated-histories

Note: Make sure git version is 2.9 or more for using --allow-unrelated-histories option

Answer 10

dumb manual solution if all other fails:

1. Download all files from first repo
2. Create a new repo
3. Clone the second repo to disk.
4. Unpack the first repo files you want to keep (new structure)
5. Push the changes to second repo.
6. Verify that you have what you want
7. Delete the files/folders from first repo that you dont want.

You lose history however