git clone fails with "fatal: Out of memory, malloc failed" error

Question

When I perform a git clone from a bare repository of my project, on a local server, I get following error message:

fatal: Out of memory, malloc failed (tried to allocate 2251896833 bytes) warning: Clone succeeded, but checkout failed. You can inspect what was checked out with 'git status' and retry the checkout with 'git checkout -f HEAD'

I tried updating my ~/.gitconfig file as stated at answer under this question, closed the git bash, restarted and retried without any results.

I ended up trying following configuration but still same results:

$ cat .gitconfig
[core]
        packedGitLimit = 1024m
        packedGitWindowSize = 1024m
[pack]
        deltaCacheSize = 1024m
        packSizeLimit = 1024m
        windowMemory = 1024m
[http]
        postBuffer = 157286400

I even tried with git gc on another machine, but not sure how to get the bare repository to garbage collect as well.

I am using git version 2.14.2.windows.1 under a 32 bits machine with Windows 7 with 4GB of RAM.

How to resolve this fatal error on git clone?

Where is the bare repo located? On a cloud service (like Github), or a service on the local network (or even just in a local server/file)? — LightCC, Oct 04 '17 at 01:15
The bare repo is located in a local server. I have a windows drive which connects to the shared folder. — Cedric Zoppolo, Oct 04 '17 at 03:48
`tried to allocate 2251896833 bytes`; which is 2,147MB, and you've set a limit of 1024MB? Do you have 2GB of available memory? — Matt Clark, Oct 04 '17 at 03:55
Yes, I have. Which values should I change? All of them? Should I change them to 2g? — Cedric Zoppolo, Oct 04 '17 at 04:06
@MattClark you can add an answer to the question so I may upvote and I will try tomorrow your solutions. As now I'm away from office. Thanks — Cedric Zoppolo, Oct 04 '17 at 04:11

Brad Peabody · Answer 1 · 2019-02-26T19:34:33.070

13

If you are on Linux and have root access, the simple workaround of adding virtual memory could be the easiest solution. On most Linux systems you can do the following (as root) to add 4GiB of virtual memory, and then try your git clone again after:

dd if=/dev/zero of=/var/swap.img bs=1M count=4096
chmod 0600 /var/swap.img
mkswap /var/swap.img
swapon /var/swap.img

This ended up being the simplest workable solution for me when I ran into this issue.

edited Feb 26 '19 at 19:34

answered Feb 18 '19 at 21:25

Brad Peabody

10,917
9
44
63

Hi, this worked for me, thank! How do I undo / reset this to clean up afterwards? – Brian Peterson Aug 20 '19 at 19:25
1

@BrianPeterson `swapoff /var/swap/img` and then delete the file. – Brad Peabody Feb 13 '20 at 19:58
swapon: /var/swap.img: swapon failed: Cannot allocate memory (512mb of actual system memory) – fr332lanc3 Jun 19 '20 at 23:03

score 12 · Accepted Answer · answered Oct 04 '17 at 05:10

Using the fact that a Git repo is self-contained and can just be copied, we only need some way to make a copy and convert from bare to non-bare for our first few attempts.

Cloning is still the best form of initial copying, if that option is possible (see work-arounds 1 and 2). If not, we can just straight copy from the server, if access is available, and convert it manually (see #3). Failing that, perhaps copying/cloning a smaller piece of the repo will work?

Work-around 1 - clone onto USB from server

Get on the server (if you have access), clone to a usb drive folder (as a non-bare repo), stick USB into target machine, and just move/copy it to the final location you want the repo. Git command should be:

git clone /path/to/bare/repo /local/repo/folder

I think this will not copy any orphan commits (those that will eventually be garbage-collected).

Work-around 2 - Clone to another PC, then copy

Clone repo onto another PC, copy/move folder somehow to target PC (USB, network connection, etc.). Similar to #1

Work-around 3 - Copy from Server, convert from bare

This assumes access to the server, perhaps through a network share rather than directly (otherwise use #1). Copy the bare repo locally, then do something like this to convert it to a non-bare repo.

Work-around 4a - Clone only one branch at a time

It appears you can clone only a single branch at a time, which may fit under the limits causing your memory issue:

git clone [url/folder] -b master --single-branch [local repo folder]

Note you could create new branches on the server that are a bit back in the history, to pull less at once, but based on the answer in the above link, you may have to jump through some hoops to remove the "single-branchedness" of your new repo.

Work-around 4b - Shallow Repo

Use --depth <depth> and/or other shallow tags to make a local repo with a limited commit history depth. From git clone --help man page:

--depth Create a shallow clone with a history truncated to the specified number of commits. Implies --single-branch unless --no-single-branch is given to fetch the histories near the tips of all branches. If you want to clone submodules shallowly, also pass --shallow-submodules.

--shallow-since= Create a shallow clone with a history after the specified time.

--shallow-exclude= Create a shallow clone with a history, excluding commits reachable from a specified remote branch or tag. This option can be specified multiple times.

--[no-]single-branch Clone only the history leading to the tip of a single branch, either specified by the --branch option or the primary branch remote’s HEAD points at. Further fetches into the resulting repository will only update the remote-tracking branch for the branch this option was used for the initial cloning. If the HEAD at the remote did not point at any branch when --single-branch clone was made, no remote-tracking branch is created.

Work-around 4c - Clone/Checkout only some files at a time

See this answer and it's link for instructions on how to pull down just part of a repo at a time. Ugly, but could work to clone part at a time.

I couldn´t get to test this solution as the PC was finally changed by another. But I consider workaruond 1 is a good solution. — Cedric Zoppolo, Oct 05 '17 at 19:05

score 3 · Answer 3 · answered Nov 19 '21 at 17:27

Also check ulimit. On AIX, I have (with no malloc failed issue):

> ulimit -a
core file size          (blocks, -c) unlimited
data seg size           (kbytes, -d) unlimited
file size               (blocks, -f) unlimited
max memory size         (kbytes, -m) unlimited
open files                      (-n) unlimited
pipe size            (512 bytes, -p) 64
stack size              (kbytes, -s) unlimited
cpu time               (seconds, -t) unlimited
max user processes              (-u) unlimited
virtual memory          (kbytes, -v) unlimited

A user having this malloc failed issue has:

> ulimit -a
core file size          (blocks, -c) 1048575
data seg size           (kbytes, -d) 131072
file size               (blocks, -f) 1048575
max memory size         (kbytes, -m) 32768
open files                      (-n) 2000
pipe size            (512 bytes, -p) 64
stack size              (kbytes, -s) 32768
cpu time               (seconds, -t) unlimited
max user processes              (-u) unlimited
virtual memory          (kbytes, -v) unlimited

User's ulimits were increased by admins; malloc failed issue resolved. — Joseph Martin, Nov 20 '21 at 05:41

score 1 · Answer 4 · answered Jun 10 '22 at 19:21

1

git config --global http.postBuffer 524288000

#If it's not enough double it! git config --global http.postBuffer 1048576000

answered Jun 10 '22 at 19:21

champion-runner

1,489
1
13
26