How to move files from a repo to another repo preserving history for moved files

Question

I have moved some files in a git repository to a new folder (foo). When I do git log (--follow) on the folder, I only see one commit which is move commit.(I used git mv for each file)

test $ git log foo/
commit 97210f4e547279ab1bdb (HEAD -> develop)
Date:   Fri Aug 17 10:41:29 2018 -0400
moving files to foo folder

test $ git log --follow foo/
commit 210f4e547279ab1bdb (HEAD -> develop)
Date:   Fri Aug 17 10:41:29 2018 -0400
moving files to foo folder

For the files in the folder, if I try git log --follow, I would see all their history. git log command, however, is the same for all of them which points to commit that I moved the files.

test $ git log --follow foo/bar.txt
commit 97210f4e547279ab1bdb (HEAD -> develop)
Date:   Fri Aug 17 10:41:29 2018 -0400
moving files to foo folder



test $ git log --follow foo/bar.txt
commit 97210f4e547279ab1bdb (HEAD -> develop)
Date:   Fri Aug 17 10:41:29 2018 -0400
moving files to foo folder

commit 12a1dad45f70ec8d62f7
Date:   Tue Oct 24 15:12:54 2017 -0400
Canned PowerShell

commit cbe9cce205640e606e13
Date:   Thu Aug 3 20:32:00 2017 -0400
Policy for no maid

Now I want to move this folder and preserve the history. I tried to use this command

git filter-branch --subdirectory-filter foo -- --all

The result is only one commit so it doesn't preserve the history of the files before I moved them to the foo folder. Even git log --follow on the files only shows one commit. I tried googling but I couldn't find anything that preserves the history before moving the files. Any help would be appreciated.

p.s: since those files got committed to upstream and there are commits after than, I prefer not to redo the move.

Answer 1

You cannot get to where you want to be, from where you are now, at least not with the current versions of Git available.

Why this is the case

Git does not have file history. Git has commits; commits are history. For each commit, as identified by its unique hash ID, a repository either has the commit, or lacks that commit. Each commit preserves a snapshot of all files, gives some metadata about who made the commit (and when and why), and identifies (by hash ID) its parent commit, or for merges, parent commits (plural).

The git log command produces the history, on demand, by traversing and displaying the commits. That is, git log shows one commit, then shows that commit's parent, and so on. At merge commits, where the history forks, git log shows both commits-and-their-parents (in some order). Using git log --graph forces Git to show the commits in an order that makes sense in terms of the graph that is the history, and at the same time, draws a crude text representation of that graph.

Given optional arguments, git log can traverse the history—the graph—but omit some of the commits, and show only commits that are particularly interesting. This is how git log filename works. Suppose you are viewing the history starting from the last commit on master. If this commit's copy of file filename is the same as the parent commit's copy, git log simply doesn't show the commit. It then moves back to the parent as usual. If the parent's parent has a different version of file filename, git log shows the parent. Then git log moves on to the parent's parent, and so on.

The result is that you see an abbreviated history, naming only those commits in which file filename has changed with respect to that particular commit's parent. If you want to call this the file's history, that's fine: but it's not what's actually in Git, which is the commit graph or commit history, it's just selected from the commit history. It looks like file history, until you rename a file, anyway.

Using git log dir/ works much the same way: it walks the commit history as usual, but suppresses printing the commit unless some file within dir/ has changed, where "changed" means comparing parent commit to child commit shows some file(s) have changed.

git log --follow does not work on directories

You cannot use git log --follow on a directory. You can only use it on one file name. The reason for this is that the implementation is a terrible hack. (This might be improved someday, in the future.) The way --follow works on a file is to use Git's rename detection.

Remember, each commit represents only a snapshot. Suppose we have a parent commit with hash G and child with hash H. The way Git decides that some file, e.g., olddir/doc.txt, was renamed between commits G and H is:

• olddir/doc.txt exists in G but not in H
• newdir/doc.txt exists in H but not in G
• the contents of olddir/doc.txt (in G) match those in newdir/doc.txt (in H), being either exactly bit-for-bit identical—this test is very fast—or within some similarity constraint (default 50% similar).

When git log --follow newdir/doc.txt< is working, it looks for a commit that touches file olddir/doc.txt between parent G and child H. If it comes across a commit that apparently renames olddir/doc.txt to newdir/doc.txt, it prints that commit—and then stops looking for olddir/doc.txt and starts looking for newdir/doc.txt instead.

It does this with exactly one full path name. If you have git log searching for directories, it never detects a rename, as it only detects renamed files.

(There was an attempt to fix / change this recently but it went awry. Hopefully there will be a better version soon.)

In any case, either your repository has the commits that have the files under the old and new names, or it doesn't. If it does not have the commits, nothing will discover them. If it does have the commits, a future Git might be able to find whole-directory renames, but current versions of Git cannot.