How to set a maximum commit count on git push

Question

I am in the process of migrating an existing SVN repository (only trunk) with over 10000 commits to a hosted Git solution (in my case BitBucket).

Converting the SVN repo to a local Git was no problem but now I want to push all revisions into an empty Git repo online.

But doing the push from TortoiseGit, it stops with this output:

git.exe push -v --progress "origin" master

Counting objects: 198817, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (83858/83858), done.
POST git-receive-pack (chunked)
Writing objects: 100% (198817/198817), 1.54 GiB | 460.00 KiB/s, done.
Total 198817 (delta 130510), reused 178506 (delta 112822)
fatal: The remote end hung up unexpectedly
fatal: The remote end hung up unexpectedly
error: RPC failed; curl 52 Empty reply from server
Pushing to https://__removed__@bitbucket.org/__removed__/__removed__.git
Everything up-to-date


git did not exit cleanly (exit code 1) (3644609 ms @ 06.10.2016 11:16:23)

I think the only solution to my problem is only to push 1000 commits at a time. But how can I limit to the first 1000 commits that aren't already on the server?

I dont want to specify each commit to push (like in this question answered). I simply want to set a maximum count.

Answer 1

I described the manual way to do this in my answer to "Git error: RPC failed; result=22, HTTP code = 404". (That answer also describes what to do if you get stuck on one large commit.)

As I mentioned in that answer and in the comments, I recommend switching from HTTPS to SSH if you can, because I haven't had this kind of problem with pushing with SSH; it seems to be limited to HTTPS.

However, if you're stuck on HTTPS, you'll have to push a smaller subset of commits at a time. As requested in your question, here's a way to automate the process (in the form of a bash script that I threw together):

#!/bin/bash
# Bisect a git push to get around HTTP POST size limits.

branch=$1
commits=$(git log --oneline | wc -l)
current=$commits

while [ $(git rev-parse origin/$branch 2>/dev/null) != $(git rev-parse $branch) ]
do
  # Git push command counts backwards, so invert the counter.
  inverse=$(expr $commits - $current)
  git push origin $branch~$inverse:$branch

  if [ $? -ne 0 ]
  then
    # If failed, bisect further, if possible.
    if [ $current > 1]
    then
      current=$(expr $current / 2)
    else
      echo "Error, could not push single commit."
      exit 1
    fi
  else
    # If successful, reset bisect.
    current=$commits
  fi
done

I don't have a repo/server handy to test this on, but it should at least get you started.

Note that this doesn't limit the commits to a certain number, but does a binary search to figure out how many commits can be successfully pushed at once.

For completeness, here's a script that pushes a specific number of commits at a time:

#!/bin/bash
# Push commits in smaller batches.

branch=$1
atonce=$2
commits=$(git log --oneline | wc -l)
current=$atonce

while [ $(git rev-parse origin/$branch 2>/dev/null) != $(git rev-parse $branch) ]
do
  # Git push command counts backwards, so invert the counter.
  inverse=$(expr $commits - $current)
  git push origin $branch~$inverse:$branch

  if [ $? -ne 0 ]
  then
    echo "Error, could not push $atonce commits."
    exit 1
  else
    current=$(expr $commits + $atonce)
  fi
done