3

A couple of years ago, a subfolder in one of my Perforce depots (e.g. //FirstDepot/LargeSubfolder/...) became too large to maintain, so I 'migrated' it into a depot of its own (e.g. //NewDepotFromLargeSubfolder/...). Perforce does not support integrations from one depot to another, so I basically checked out the files, added them to the new depot and deleted them from the subfolder.

Now I am about to migrate the entire Perforce server to a number of git repositories. The git p4 procedure works very well, I already migrated other, unrelated Perforce depots to git successfully (including their full file history using @all - but then, those depots were never 'migrated' within Perforce in the way described...).

The problem is that I do not want to lose the file history underneath //FirstDepot/LargeSubfolder/.... There are a couple thousand changelists in there (plus a couple thousand more at //NewDepotFromLargeSubfolder/...) and many of them contain very valuable information.

I tried to migrate the file history from the first location and then 'stack' the history of the second location on top of it:

host:~ user$ mkdir SomeTempFolder
host:~ user$ cd SomeTempFolder
host:SomeTempFolder user$ export P4PORT=perforce:1666
host:SomeTempFolder user$ git p4 clone //FirstDepot/LargeSubfolder@all gitRepoFolder
Importing from //FirstDepot/LargeSubfolder@all into gitRepoFolder
Initialized empty Git repository in /Users/user/SomeTempFolder/gitRepoFolder/.git/
Import destination: refs/remotes/p4/master
Importing revision 26776 (99%)
Ignoring apple filetype file //FirstDepot/LargeSubfolder/SomePath/SomeIconFile.icns
Importing revision 26778 (100%)

So apart from a message about a Mac OS X icon file being ignored, the first bit works. The second part fails however:

host:SomeTempFolder user$ git p4 clone //NewDepotFromLargeSubfolder@all gitRepoFolder
Importing from //NewDepotFromLargeSubfolder@all into gitRepoFolder
Reinitialized existing Git repository in /Users/user/SomeTempFolder/gitRepoFolder/.git/
Import destination: refs/remotes/p4/master
Importing revision 26777 (0%)
Ignoring apple filetype file //NewDepotFromLargeSubfolder/SomePath/SomeIconFile.icns
Importing revision 27142 (33%)
Ignoring apple filetype file //NewDepotFromLargeSubfolder/AnotherPath/Some AppleScript file.scpt
Ignoring apple filetype file //NewDepotFromLargeSubfolder/AnotherPath/Another AppleScript file.scpt
Importing revision 27620 (100%)

fast-import failed: warning: Not updating refs/remotes/p4/master (new tip <some long git ID> does not contain <another long git ID>)
git-fast-import statistics:
<loads of git statistic>

Running git p4 with --verbose seems to indicate that the actual importing of Perforce revisions completes successfully, but then something goes wrong which I don't understand:

Importing revision 27620 (100%)commit into refs/remotes/p4/master
Opening pipe: ['p4', '-G', '-x', '-', 'print']
Path/of/the/file/in/the/last/Changelist

Reading pipe: ['git', 'config', '--bool', 'git-p4.importLabels']
Traceback (most recent call last):
  File "/usr/libexec/git-core/git-p4", line 3287, in <module>
    main()
  File "/usr/libexec/git-core/git-p4", line 3281, in main
    if not cmd.run(args):
  File "/usr/libexec/git-core/git-p4", line 3155, in run
    if not P4Sync.run(self, depotPaths):
  File "/usr/libexec/git-core/git-p4", line 3030, in run
    die("fast-import failed: %s" % self.gitError.read())
  File "/usr/libexec/git-core/git-p4", line 106, in die
    raise Exception(msg)

Exception: fast-import failed: warning: ... (as above) ...

The migration of the second part works fine when done into a new git repository.

After reading `git-p4 clone` fails "new tip ... does not contain ...", I retried the migration in a working folder without spaces; same result. I'm working on Mac OS X btw, but the Perforce server is running on a Debian 7.5 virtual machine.

My questions:
+ Does anyone have some advice how to find out what's wrong here ?
+ Is it generally possible to 'stack' chunks of revision history ?
+ Can anyone think of an alternate approach (apart from having two git repos) ?

I'm not a git expert at all, but so far, I could pretty much just throw stuff at it and git did an excellent job of figuring out what I want to do; also the Reinitialized existing Git repository message seems to indicate that I'm not totally off.

Community
  • 1
  • 1
ssc
  • 9,528
  • 10
  • 64
  • 94
  • Perhaps you can look in your Perforce server's log to see exactly what command is being run at the point where your tool fails. If you have a new-enough Perforce server, and you run your Perforce server with the 'server=4' trace setting, all error messages which are sent to client software are also logged to the server's log, which should help you locate what exact error message your tool is receiving. – Bryan Pendleton Jan 24 '15 at 18:10
  • The server version is `P4D/LINUX26X86_64/2013.1/678655 (2013/07/29)`. Tried starting `p4d` with `-v server=4` (although the [docs](http://answers.perforce.com/articles/KB_Article/Perforce-Server-Trace-Flags) say 3 is highest) and re-ran the migration attempt: No error message whatsoever, just the usual output. – ssc Jan 24 '15 at 18:31
  • The server's output won't be sent to your screen; it will be written to a log file in the server's filesystem. The name of that log file is given with the P4LOG setting for the server. If you don't have a P4LOG definition for your server, it may not be writing a log, so you should configure a log and then rerun your test and look in that log. – Bryan Pendleton Jan 25 '15 at 17:14
  • Thanks, I am aware of that; by 'just the usual output' I meant messages starting with `Perforce server info:` in `/var/log/perforce/log`. I should have expressed that more clearly. I'm not sure I'm looking at a Perforce server issue here; after all, all the server gets is a long list of queries for changelist descriptions... – ssc Jan 25 '15 at 20:12

1 Answers1

1

The migration of the second part works fine when done into a new git repository

That could be the basis of an acceptable workaround here: have to separate Git repo, and chose a technique from "How do you merge two git repositories?" to get one final repo.

The git filter-branch approach is detailed in "Combining multiple git repositories".

(You also have a simpler approach based on merge, also described here: The OP ssc selected "Merge two Git repositories without breaking file history")

Community
  • 1
  • 1
VonC
  • 1,262,500
  • 529
  • 4,410
  • 5,250
  • Thanks for that! [This](http://stackoverflow.com/a/14470212) did the trick. Enjoy your bounty :-) – ssc Jan 28 '15 at 14:52
  • @ssc Great! I have include the link to the solution you selected in the answer for more visibility. – VonC Jan 28 '15 at 14:58