7

I have looked at several threads addressing this.

Combining multiple git repositories

Combining multiple git repositories having a space in their name

I also looked at the git filter-branch manpage.

update

I have changed to a 2 script system:

#!/bin/bash
git filter-branch --index-filter '~/doit.sh' HEAD

and doit.sh

#!/bin/bash
git ls-files -s | \ 
    sed "s-\t-&data/perl_modules/-" | \ 
    GIT_INDEX_FILE="$GIT_INDEX_FILE.new" git update-index --index-info && \
    mv "$GIT_INDEX_FILE.new" "$GIT_INDEX_FILE"

This avoids the previous error, but now gets this (replaced path with [...]):

Rewrite c35e4ef0626fb2045f57fa5b605e7663b8d06196 (1/10977)mv: cannot stat `[...]/.git-rewrite/t/../index.new': No such file or directory
index filter failed: ~/doit.sh

When I run the 

ls-files- s | sed ... | git update-index ...

I get the index file it should generate. As well when I change the doit.sh file to output the result of sed instead of piping it to git update-index it appears to produce the proper output... it seems git update-index is simply not creating the file when run under --index-filter....

Update again:

When I change

mv "$GIT_INDEX_FILE.new" "$GIT_INDEX_FILE"

to

mv "$GIT_INDEX_FILE.new" "$GIT_INDEX_FILE" || true

It fails the first mv, but all the others (so far) are working.

All of this culminated in this script:

git filter-branch --index-filter \
    'git ls-files -s | sed "s-\t\"*-&data/perl_modules/-" |
            GIT_INDEX_FILE=$GIT_INDEX_FILE.new \
                    git update-index --index-info &&
     mv $GIT_INDEX_FILE.new $GIT_INDEX_FILE' HEAD

Theoretically this script should take all the files in the repository, shove them into data/perl_modules/, and rewrite history so that it appears the files have always been in that directory.

However I get this error:

fatal: ambiguous argument 'ls-files': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions

Not sure how to proceed, I don't understand the script well enough to debug it, and it is taken directly from the git filter-branch manpage.

I have tried this both before and after manually moving the files to the subdir in case it required they be moved, or required they not be moved.

Community
  • 1
  • 1
Exodist
  • 629
  • 5
  • 15
  • 1
    Brett Randall has now submitted a patch to the Git mailing list which can be inspected at http://thread.gmane.org/gmane.comp.version-control.git/270407 – Philip Oakley Jun 01 '15 at 21:24

2 Answers2

15

There were a couple problems.

1) git filter-branch --index filter was not using the '...' properly.

I am not sure why this is, a bug with git-filter, an environment problem? who knows. But I was able to fix it by moving everything out of the '...' and putting it into a script file. I then called the script inside the ''

git filter-branch --index-filter '~/doit.sh' HEAD

And doit.sh:

#!/bin/bash

git ls-files -s | \ 
    sed "s-\t-&data/perl_modules/-" | \ 
    GIT_INDEX_FILE="$GIT_INDEX_FILE.new" git update-index --index-info

mv "$GIT_INDEX_FILE.new" "$GIT_INDEX_FILE" || true

2) The first commit was empty, so there was no index.new to mv

This git repo was imported from svn using git-svn. As such the first commit was completely empty simply saying initial svn repo initialization. As such there were no files moved, and no index.new to move. This was solved by adding || true to the mv command. Note however that if more than one mv command fails you have other problems.

Exodist
  • 629
  • 5
  • 15
  • if the first commit is the one with issues it would be better (in my opinion) to replace "HEAD", by "SHA2 of second commit..HEAD". – rufo Apr 08 '14 at 15:06
  • 1
    @rufo As I have commented elsewhere, it painfully turns out that replacing HEAD with REV2..HEAD is _not_ a general solution. – Ryan Jul 22 '14 at 19:18
  • 4
    Solution for me was to add `if [ -f "$GIT_INDEX_FILE.new" ]; then mv...EX_FILE"; fi` -- for some reason there was a legitimate commit in svn (on a build.info file) that did not make it into git after the git svn clone. And for whatever reason removing empty commits did not pick this one up. – Ryan Jul 23 '14 at 20:06
  • 1
    @Ryan Can you tell me where exactly you put that? I have exactly the same problem, it's frustrating. –  May 17 '16 at 15:26
  • 1
    Like this: `if [ -f "$GIT_INDEX_FILE.new" ]; then mv "$GIT_INDEX_FILE.new" "$GIT_INDEX_FILE"; fi` – Patrick Beard Jul 12 '17 at 02:24
1

looks like you have \ missing in some of the multiline breaks.

Adam Dymitruk
  • 124,556
  • 26
  • 146
  • 141
  • Tried turning it all into one line to avoid need for \, same problem. – Exodist Oct 17 '11 at 21:05
  • how about other \ symbols. Did you try and escape them? Also, * may be causing issues. Start with a simpler script than what you have and add complexity incrementally (use of \ and *). – Adam Dymitruk Oct 17 '11 at 21:59
  • That is not the issue, see my latest update. I replaced everything in the single quotes with a call to script containing what they used to have. Now I get a different error. – Exodist Oct 17 '11 at 22:10
  • Interesting. I haven't had issues with filter-branch before like this. Have you checked with the guys on the #git irc channel? I usually get great help there. Meanwhile, I'll try to recreate this on my own. – Adam Dymitruk Oct 17 '11 at 22:25
  • the #git channel on freenode had little interest in my question. Maybe I asked at a bad time? – Exodist Oct 17 '11 at 22:27
  • This is a repo that was imported from svn if that makes a difference. – Exodist Oct 17 '11 at 22:28
  • Oh, also I am using zsh, maybe the interpolation is having an effect? – Exodist Oct 17 '11 at 22:29
  • yes, the shell can have a different interpretation and escaping. Try omitting the script file. Just concentrate on update index not working. Do something dead simple that exercises that.. – Adam Dymitruk Oct 17 '11 at 22:42
  • 1
    I have actually completely solved my problem at this point. I simply cannto post the answer for another several hours due to my reputation. I have a very pretty and nice detailed explanation of all problems and solutions, and will post them when I am allowed. – Exodist Oct 17 '11 at 22:46