GitLab Secondary Geo code push fails with Internal API unreachable

Question

Code size is 6GB. While trying to push to secondary geo node from git client system, it fails with below error.

git push -u origin master
remote:
remote: You're pushing to a Geo secondary! We'll help you by proxying this
remote: request to the primary:
remote:
remote:   ssh://git@ssh.gitlab-primary.domain.com:443/upload_grp/upload_from_secondary_geo.git
remote:
Enumerating objects: 175198, done.
Counting objects: 100% (175198/175198), done.
Delta compression using up to 16 threads
Compressing objects: 100% (114761/114761), done.
Writing objects: 100% (175198/175198), 1.40 GiB | 103.54 MiB/s, done.
Total 175198 (delta 50871), reused 175198 (delta 50871)
remote:
remote: ========================================================================
remote:
remote: Internal API unreachable
remote:
remote: ========================================================================
remote:
fatal: the remote end hung up unexpectedly
fatal: the remote end hung up unexpectedly

Is it something related with network related problem? Any direction to fix the problem would help a lot.

VonC · Answer 1 · 2020-02-28T07:27:53.497

1

That looks like the new way to display an internal GitLab server error, as introduced with GitLab 12.5+ (Nov. 2019)

That needs to be checked at the GitLab server level, to see if its log includes any clue.
That being said, make sure the commit pushed itself if not 6GB... that would likely be rejected on its own (too big)

The OP Kishore confirms in the comments:

I tried with 6-GB code where as it didn’t work, Later 662M & 1.4-GB code I could able to push successfully.

The solution remains to break your large commit in multiple smaller ones.

edited Feb 28 '20 at 07:27

answered Feb 28 '20 at 05:28

VonC

1,262,500
529
4,410
5,250

/var/log/gitlab/gitlab-shell/gitlab-shell.log, /var/log/gitlab/gitlab-rails/application.log /var/log/gitlab/gitlab-rails/production.log doesn't show any error messages. – user4948798 Feb 28 '20 at 05:58
i could able to push small code(662M) successfully to remote repository. – user4948798 Feb 28 '20 at 07:04
@Kishore then is the current failed commit much larger than that? – VonC Feb 28 '20 at 07:05
Initially I tried with 6-GB code where as it didn’t work, Later 662M & 1.4-GB code I could able to push successfully. – user4948798 Feb 28 '20 at 07:23
@Kishore OK. I have edited the answer with a possible solution then. – VonC Feb 28 '20 at 07:28
I feel some bug in GitLab latest version - `12.7.5-ee`. Now i tried to push 2.7-GB code, but it failed – user4948798 Feb 28 '20 at 08:53
@Kishore Do you have a `client_max_body_size` as in https://gitlab.com/gitlab-org/gitlab-foss/issues/17917? – VonC Feb 28 '20 at 08:56
`client_max_body_size` not configured in `gitlab.rb` file. – user4948798 Feb 28 '20 at 09:08

GitLab Secondary Geo code push fails with Internal API unreachable

1 Answers1