75

Let's say I have a repo that includes this directory structure:

repo/
  blog/
    _posts/
      some-post.html
  another-file.txt

I want to move _posts to the top level of the repo, so the structure will look like this:

repo/
  _posts/
    some-post.html
  another-file.txt

This is simple enough with git mv, but I want to make the history look as though _posts always existed at the root of the repo, and I want to be able to get the entire history of some-post.html via git log -- _posts/some-post.html. I imagine I can use some magic with git filter-branch to accomplish this, but I haven't figured out exactly how to do that. Any ideas?

mipadi
  • 398,885
  • 90
  • 523
  • 479

4 Answers4

105

You can use the subdirectory filter to achieve this

 $ git filter-branch --subdirectory-filter blog/ -- --all

EDIT 1: If you don't want to effectively make _posts the root, use a tree-filter instead:

 $ git filter-branch --tree-filter 'mv blog/_posts .' HEAD

EDIT 2: If blog/_posts did not exist in some of the commits, the above will fail. Use this instead:

 $ git filter-branch --tree-filter 'test -d blog/_posts && mv blog/_posts . || echo "Nothing to do"' HEAD
slm
  • 15,396
  • 12
  • 109
  • 124
artagnon
  • 3,609
  • 3
  • 23
  • 26
  • 4
    It's also *much* faster to use `--index-filter`, since it doesn't have to check out the tree. – Cascabel Jun 29 '10 at 17:46
  • 7
    Yeah index-filter is faster, but it won't work because the commands shown do _not_ affect the index. You need to do index manipulations only if you want to use index-filter (e.g. `git rm --cached` instead of `rm`) – sehe Mar 26 '11 at 22:48
  • Can I keep tags too? It looks like they are gone in my case. – Michael Jun 09 '17 at 02:10
  • 1
    `git filter-branch --index-filter 'git read-tree --prefix=/ $GIT_COMMIT:_posts; git rm -r --cached _posts' -- --all`. Add `--tag-name-filter cat` if your tags aren't signed and you want to move them/invalidate the old ones. – jthill Dec 11 '17 at 20:46
  • For those that are here just for copy-paste: remember to `git push --force --all` for all the branches. Otherwise you can end up with funny situations. – SiliconMind Oct 15 '19 at 16:51
  • No need to make the last command so complicated. To move existing folder somewhere else regardless if it exists in all commits, just run `git filter-branch --tree-filter 'git mv -k blog/_posts .'` Index filter will not work in that case, as you are modifying the tree objects as well (and you need to have them checked-out), not just the index. The `-k` switch will take care of the "nonexistent" files in some commits. – petrpulc Nov 24 '22 at 10:32
39

While Ramkumar's answer is very helpful and worthwile, it will not work in many situations. For example, when you want to move a directory with other subdirectories to a new location.

For this, the man page contains the perfect command:

git filter-branch --index-filter \
  'git ls-files -s | sed "s-\t\"*-&NEWSUBDIR/-" |
   GIT_INDEX_FILE=$GIT_INDEX_FILE.new \
   git update-index --index-info &&
   mv "$GIT_INDEX_FILE.new" "$GIT_INDEX_FILE"' HEAD

Just replace NEWSUBDIR with your desired new directory. You can also use nested dirs like dir1/dir2/dir3/-"

theduke
  • 3,027
  • 4
  • 29
  • 28
  • 12
    And since it's not immediately obvious from looking at that command or the resulting errors, the \t doesn't work on os x's version of sed. There's lots of ways around that, but perhaps the quickest is to delete the \t and replace it with a literal tab by typing ctrl-v, . – Jeremy Huiskamp Jan 06 '14 at 21:04
  • 4
    How do you specify the original folder, or does this just move the entire branch? When I try to run this from a folder I'd like to move I get `You need to run this command from the toplevel of the working tree` – joshcomley Jan 08 '15 at 14:41
  • 1
    What does the `sed` command do? I'm trying this on Windows and need an alternative. – Lucius Feb 12 '15 at 13:11
  • Thanks, this helped me a lot! @Lucius, the sed command is much more clear if you replace the "&" with another \t and the "-"s with an @. – Will Mar 02 '15 at 16:36
  • 1
    Brilliant. But yes the sed is confusing so try the second line alone to test it. I.e. to remove unnecessary top level directories I did a simple `git ls-files -s | sed "s-\tdir1/dir2/dir3/-\t-"` – KCD Jun 18 '15 at 01:37
  • I tried so many answers on the internet, this is the only one working - thx a lot! – dr0i Jan 25 '16 at 14:39
  • 2
    If you're filtering a commit that effectively deletes all files you end up with `git update-index` not creating the file "$GIT_INDEX_FILE.new" and thus the mv command fails. I ended up with `test -f \$GIT_INDEX_FILE.new && mv \$GIT_INDEX_FILE.new \$GIT_INDEX_FILE || touch \$GIT_INDEX_FILE` inside the filter-branch script. – Knut Forkalsrud Jan 28 '16 at 03:25
  • @KnutForkalsrud I get the error of index.new': No such file or directory, as this git repository was a result of an svn-migration, there is no initial commit. Could you share your command, or is it just insert that into the filter, with no quotations? –  May 17 '16 at 14:58
  • @Jägermeister - just replace the "mv" command in theduke's example with the "test && mv || touch" combination from my comment. – Knut Forkalsrud Jun 19 '16 at 18:59
  • Something about using `-` as a separator confused my sed, so I had to change that like so: `sed "s:\t\"*:&NEWSUBDIR/:"`. – polm23 Apr 26 '17 at 06:39
  • 1
    As noted, BSD/macos sed does not support `\t`, and the pattern fails on paths with hyphens in their name -- so I wrote a cross platform solution that uses awk; `git ls-files -s | awk '{ sub(/\t/, "\tsubdir/"); print}' ` full usage: set DEST to desired subdir `DEST="subdir/path/" git filter-branch -f --index-filter 'git ls-files -s | awk -v prefix="$DEST/" "{ sub(/\t/, \"\t\" prefix); print}" GIT_INDEX_FILE=$GIT_INDEX_FILE.new git update-index --index-info && "$GIT_INDEX_FILE.new" "$GIT_INDEX_FILE"' HEAD` – briceburg Apr 27 '17 at 20:51
  • @briceburg The `mv` is missing between `&&` and `"$GIT_INDEX_FILE.new"`. Other than that, it works perfectly! – gablin Mar 08 '18 at 12:00
  • Fails with the following error Rewrite 7576b38152b393793b1c9ec3df0ff86685f95236 (1/8246) (0 seconds passed, remaining 0 predicted) mv: cannot stat '/tmp/controller/.git-rewrite/t/../index.new': No such file or directory index filter failed: git ls-files -s | sed "s-\t\"*-&opendaylight/blueprint/-" | GIT_INDEX_FILE=$GIT_INDEX_FILE.new git update-index --index-info && mv "$GIT_INDEX_FILE.new" "$GIT_INDEX_FILE" – askb Aug 29 '18 at 02:59
  • 1
    @askb: You can skip bad commits with appending `; /bin/true`. See [this excellent post](https://stackoverflow.com/a/46677910/4566599). – Roi Danton Sep 11 '18 at 12:20
  • For those Windows users who stumbled on this answers like me and wonder how to make it work on Windows: consider using WSLS and run the command from linux. Just make sure that your repository is checked out with AutoCRLF to false or else the git command in linux is going to think all files have changes and will block the execution. – Etienne Maheu Apr 08 '20 at 01:53
  • Running under Cygwin on Windows, the above didn't work. Eventually, my issue was that my subdir contains dashes (e.g. "sect-03"), which is the separator used in the `sed` section. Replacing `sed "s-\t\"*-&sect-03/-"` with `sed "s/\t\"*/&sect-03\//"` did the trick. – AVIDeveloper Apr 18 '20 at 20:17
  • I also ran into a problem with the `-` being used as the sed separator when I was trying to move a directory that had a hyphen in the name. It took me forever to realize what was going on. I'm guessing they went away from the usual sed `/` separator because that's common to see that character used in paths. It might be smarter to use an even less commonly used character such as `^` instead: `sed "s^\t\"*^&NEWSUBDIR/^"` – dougg3 Apr 10 '23 at 18:03
2

Created a generic script for arbitrary moves/renames: https://gist.github.com/xkr47/f766f4082112c086af63ef8d378c4304

Examples:

git filter-mv 's!^!subdir/!'

➜ moves all files to a subdirectory "subdir/" in all commits of the current branch

git filter-mv 's!^foo/bar.txt$!foo/barbar.txt!'

➜ renames foo/bar.txt to foo/barbar.txt in all commits of the current branch

Jonas Berlin
  • 3,344
  • 1
  • 27
  • 33
1

git filter-branch is discouraged. You can use git filter-repo is much more convenient:

git filter-repo --path-rename blog/_posts:_posts

See here.

Also note that installation somehow is broken, and you have to install from pip and add git-filter-repo to $PATH

Guillaume Jacquenot
  • 11,217
  • 6
  • 43
  • 49
Constantine
  • 61
  • 6