1

I've been trying to import libre core (https://github.com/LibreOffice/core) to an azure devops repo. The url I am closing is: https://github.com/LibreOffice/core.git

The import keeps failing with error:

Oops! Your import of https://github.com/LibreOffice/core.git repository failed due to The commit object 657924e4d73d6d501c9a3ceaf62e29b8f243cead was rejected: Commit parse failed due to author identity failed to parse: Andre Fischer<andre.f.fischer <Andre Fischer<andre.f.fischer@oracle.com>>

Maybe it's due to the extra < in the author? So I've tried importing this to another github repo and that still fails which suggests it's not just an azure devops problem.

I'm able to clone the repo locally but when I try and push that to azure devops I get the same error. I've also tried looking at potentially finding the commit and changing the author but because there's over 400,000 commits and this one is fairly early on my computer crashes when trying out various methods I've read here

Any advice on how to work around this would be really great!!

Added below output from fast export and import command:

Alloc'd objects:    4615000
Total objects:      4614971 (    426688 duplicates                  )
      blobs  :      1635681 (         0 duplicates    1511600 deltas of    1616683 attempts)
      trees  :      2531286 (    426688 duplicates    2306584 deltas of    2490676 attempts)
      commits:       448004 (         0 duplicates          0 deltas of          0 attempts)
      tags   :            0 (         0 duplicates          0 deltas of          0 attempts)
Total branches:           1 (         1 loads     )
      marks:     1073741824 (   2083685 unique    )
      atoms:          87634
Memory total:        246981 KiB
       pools:         30653 KiB
     objects:        216328 KiB
---------------------------------------------------------------------
pack_report: getpagesize()            =       4096
pack_report: core.packedGitWindowSize = 1073741824
pack_report: core.packedGitLimit      = 35184372088832
pack_report: pack_used_ctr            =     429158
pack_report: pack_mmap_calls          =      32709
pack_report: pack_open_windows        =          1 /          9
pack_report: pack_mapped              =  214272731 / 9353540391
---------------------------------------------------------------------

And the command I'm running is:

git fast-export master| sed -e 's/<Joerg Skottke \[jsk\] jsk@openoffice.org>>/<jsk@openoffice.org>/' -e 's/Andre Fischer<andre.f.fischer <Andre Fischer<andre.f.fischer@oracle.com>>/Andre Fischer <andre.f.fischer@oracle.com>/' -e 's/Andre Fischer<Andre.W.Fischer <Andre Fischer<Andre.W.Fischer@Sun.COM>>/Andre Fischer <andre.f.fischer@oracle.com>/'  -e 's/Gregor Hartmann<gh <Gregor Hartmann<gh@openoffice.org>>/Joerg Skottke <jsk@openoffice.org>/' | (cd ../new-repo && git fast-import)
Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
itadvicehelpsdf
  • 147
  • 1
  • 3
  • 10
  • 1
    Do you really need the entire history? May be `git clone --depth 1000` would be enough? – phd Mar 19 '20 at 17:24
  • Forgive my ignorance but is it possible to do that? Would I still get all the files in their latest form and just not have history? Would there be any other consequences of this? – itadvicehelpsdf Mar 20 '20 at 10:20
  • https://git-scm.com/docs/git-clone#Documentation/git-clone.txt---depthltdepthgt, https://stackoverflow.com/search?q=%5Bgit%5D+shallow+clone – phd Mar 20 '20 at 13:18

1 Answers1

1

Most Git hosting providers run with transfer.fsckObjects on all their repositories so that any invalid, corrupt, or malicious commit is rejected immediately. This prevents people from pushing objects using broken tools or exploiting security vulnerabilities in older clients.

However, that tends to cause problems for well-known repositories with corrupted commits, such as this LibreOffice one. The extra angle brackets are definitely not allowed here and Git is rejecting the commit due to them.

Some hosting providers can relax one or more checks for an import of an existing, well-known repository, so it would be worth asking Azure DevOps support if that's a thing they can do. If not, your best best would be to use git fast-export to export the repository, modify the stream to fix the author (and if necessary, committer) identities, and then import it into a fresh repository using git fast-import. Note that doing so will change the object IDs of any commits starting at the invalid one and invalidating any subsequent tag signatures.

You can do this with something like the following:

$ git init ../new-repo
$ git fast-export | \
  perl -pe 's/(author|committer) Andre Fischer<andre.f.fischer <Andre Fischer<andre.f.fischer@oracle.com>>/$1 Andre Fischer <andre.f.fischer@oracle.com>/' | \
  (cd ../new-repo && git fast-import)
bk2204
  • 64,793
  • 6
  • 84
  • 100
  • Thanks for the response - I've contacted Azure DevOps to see if they can help already (they couldn't). But I will let them know about relaxing the import. Regarding your second suggestion do you mind expanding a bit more? How would I modify the stream to fix the author? – itadvicehelpsdf Mar 20 '20 at 10:17
  • 1
    I've provided an example of an invocation you can use. – bk2204 Mar 20 '20 at 19:58
  • Thanks for that but still get an error message of :"fatal: Missing space before < in ident string: Gregor Hartmann> 1282910542 +02000" have tried to run perl -pe 's/(author|committer) Gregor Hartmann>/$1 Gregor Hartmann /' but am still getting the same error – itadvicehelpsdf Mar 23 '20 at 12:13
  • 1
    You'll probably want to save the `git fast-export` output to a file and fix the various problems with `perl -pi -e …`. It looks like your string is slightly different than the one the error message produced, so that may be why it's not working. – bk2204 Mar 23 '20 at 22:24
  • Unfortunately this doesn't allow me to reupdate my repo with changes on teh libreoffice one as this modifies the commits history – itadvicehelpsdf Mar 30 '20 at 15:15
  • 1
    Any fix you make for this is necessarily going to rewrite the commit history. If Azure DevOps can't let you upload this as it is, then you could try another provider, like GitHub, if you are unwilling to modify the history. – bk2204 Mar 30 '20 at 22:21
  • Fair enough thanks for the info. So I've managed to get the export to complete but my repo folder is empty. The output it gives has been added to the main post- you know why that may be? – itadvicehelpsdf Mar 31 '20 at 10:38
  • `git fast-export` only writes to the repository, not the working tree. If you want to check out a branch, like `master`, you need to run `git checkout master`. – bk2204 Mar 31 '20 at 22:40