filter-branch to remove extra directory

Question

I'm migrating several Subversion repositories to Git, but they were set up a bit weird: the standard trunk/branches/tags structure lives within several subdirectories of the repository. /branches and /tags are both empty, so the git-svn import ended up with only the trunk directories.

svn/
  proj1/
    trunk/
      src/
  proj2/
    trunk/
      src/

I would like to use git filter-branch to remove the extra trunk folders but leave the rest intact:

svn/
  proj1/
    src/
  proj2/
    src/

Based on the last example in the documentation, this is what I have so far:

git filter-branch --index-filter \
    'git ls-files -s | sed "s-/trunk/-/-" |
        GIT_INDEX_FILE=$GIT_INDEX_FILE.new \
            git update-index --index-info &&
     mv "$GIT_INDEX_FILE.new" "$GIT_INDEX_FILE"' HEAD

It seems to me that the sed is doing its job, as this input:

100644 fa3dcf79717e9aca85ad745078a2fb2a2ce2b900 0       proj1/trunk/src/file1.txt

Produces this output:

100644 fa3dcf79717e9aca85ad745078a2fb2a2ce2b900 0       proj1/src/file1.txt

However, when I run the whole command, I get the following error:

Rewrite e00c119cfb755f741dc0e17c36d36bc2ddd27562 (1/42)mv: cannot stat `/c/dev/R
epo/.git-rewrite/t/../index.new': No such file or directory
index filter failed: git ls-files -s | sed "s-/trunk/-/-" |
            GIT_INDEX_FILE=$GIT_INDEX_FILE.new \
                git update-index --index-info &&
         mv "$GIT_INDEX_FILE.new" "$GIT_INDEX_FILE"

Any ideas? I'm using git version 1.7.3.1.msysgit.0 on Windows.

Update: Now using git version 1.9.0.msysgit.0, a similar situation yields a different error (note .git-rewrite does not exist before executing the command):

rm: cannot remove `c:/Path/to/svn/.git-rewrite/revs': Permission denied
rm: cannot remove directory `c:/Path/to/svn/.git-rewrite': Directory not empty

The same fix ultimately worked.

score 9 · Accepted Answer · edited May 23 '17 at 10:30

9

Figured it out based on a comment here. It seems to be an issue with how git-svn represents "empty" (directory-only) commits. It works fine if I start at a normal commit:

git filter-branch --index-filter \
    'git ls-files -s | sed "s-/trunk/-/-" |
        GIT_INDEX_FILE=$GIT_INDEX_FILE.new \
            git update-index --index-info &&
     mv "$GIT_INDEX_FILE.new" "$GIT_INDEX_FILE"' 83df928c..HEAD

edited May 23 '17 at 10:30

Community

1
1

answered Apr 15 '11 at 15:34

dahlbyk

75,175
8
100
122

Ah, interesting. That's good to know. I guess `git update-index` doesn't actually create an index file if it gets nothing from `git ls-files`; it doesn't seem to be able to produce an empty index from empty input. – Brian Campbell Apr 15 '11 at 15:48
1

Did nothing for me. Looking for a more general solution. – Ryan Jul 22 '14 at 19:16
I'm sorry to hear that my specific answer to my specific question doesn't answer your more general question. – dahlbyk Jul 24 '14 at 03:58
1

In case anyone has a similar situation to me, it was slightly different from OP: the initial commit in my git repository was empty (e.g. commited using 'git commit --allow-empty'). Running filter-branch from the commit after the empty commit to HEAD fixed the issue: e.g. FIRST_COMMIT..HEAD. Empty commits that are not the initial commit didn't seem to cause any issues. – vman Jan 30 '15 at 17:20
I have exactly the same issue, however I am new to git, and I wonder if that `83df928c..HEAD` is a part of the command you propose for this problem, or is it something very specific to your problem? – May 17 '16 at 09:18
83df928c is just a reference to the first "real" commit - replace that with the SHA of your first non-empty commit, and you should be good to go. More generally, `foo..bar` means "commits from `foo` to `bar`, and `HEAD` is essentially "here" (what's currently checked out). So `83df928c..HEAD` means "commits from `83df928c` to here". – dahlbyk May 17 '16 at 15:55
Worked - but I cannot run it a second time - `rm: cannot remove .git-rewrite’: Directory not empty` any idea what the problem is? – Benjamin Peter Oct 15 '19 at 08:31
1

@BenjaminPeter try `rm -rf .git-rewrite`, the temp folder where `filter-branch` stores stuff. – dahlbyk Oct 16 '19 at 18:41

score 1 · Answer 2 · answered Apr 15 '11 at 14:55

Hmm. I can't see anything wrong with what you're running, and it works fine for me in an example repository:

$ git init
Initialized empty Git repository in /Users/bcampbel/tmp/git-subdir/.git/
$ mkdir -p proj1/trunk
$ echo "hello" > proj1/trunk/test.txt
$ git add .
$ git commit -m "initial commit"
[master (root-commit) cd431f3] initial commit
 1 files changed, 1 insertions(+), 0 deletions(-)
 create mode 100644 proj1/trunk/test.txt
$ echo "foo" > proj1/trunk/another.txt
$ git add .
$ git commit -m "add a file"
[master 84139f2] add a file
 1 files changed, 1 insertions(+), 0 deletions(-)
 create mode 100644 proj1/trunk/another.txt
$ git filter-branch --index-filter \
>     'git ls-files -s | sed "s-/trunk/-/-" |
>         GIT_INDEX_FILE=$GIT_INDEX_FILE.new \
>             git update-index --index-info &&
>      mv "$GIT_INDEX_FILE.new" "$GIT_INDEX_FILE"' HEAD
Rewrite 84139f26c7921721190b3c5dd57e2b49a41034aa (2/2)
Ref 'refs/heads/master' was rewritten
$ ls proj1/
another.txt test.txt

It appears that for some reason, the index.new file did not get written out on your system, or could not be read by the mv command. I am not sure why this would be, but I wonder if it may be due to the fact that you're using msysgit on Windows. Do you have a Linux box or Mac that you could try it on?

filter-branch to remove extra directory

2 Answers2

Linked