1

I see a reproducible speed degradation on git-fetch operation here. Environment: Win10x64, latest Git-For-Windows (2.37.1), having a repo on one SSD and adding another Git repository as remote, which is located on another SSD (they have the same root, I wanted to merge things over). Both are professional SSDs on the same machine and with this gear, the expected fetching speed should be at least 100MiB/s.

The result: far from it. For the first 60% of the reported fetch process, the speed is shown as jumping between 5 and 50 MiB/s. And then, after roughly 1GiB, it totally breaks down to less than 2MiB/s, and stays like this for hours.

So I checked Git index and commit is very slow and that is not helpful - I see more trace output with those vars but once the transfer has started, the log stops printing.

And so I made some traces with the "Process Monitor" and I see some patterns. I see three git.exe tasks, a) the caller, b) receiver, c) sender, and a git-upload-pack process. And it seems like the CPU usage is below one percent for all of them. When I check the procmon output, I see the receiver doing dozens or even hundreds of WriteFile syscalls per second (BUT: using a window of 4kB). And almost no syscall activity from others. Once a minute (!), the other git.exe instance wakes up and does a series of ReadFile actions with a buffer of one MB for each, and it goes silent again.

Now I am wondering – is this some Windows/MSYS specific data stream issue? Why is the receiver using such small reading windows? Or are they small just because data is being delivered so slow? If so, why is it so slow?

Prequel to this story: the source repository is very large (dozens of GBs) and has a wild mix of BLOBs of different sizes. So, before this we had to set the option core.bigFileThreshold to 1m. Without this, the fetch performance was even creepier, getting stuck at "remote: compressing ..." and died with out-of-memory error after a half hour or so. Maybe the issues are related.

PasterOfMuppets
  • 165
  • 9
  • Did you check the reported bandwidth from the OS to see if it matches up with what Git reports? – TTT Aug 11 '22 at 03:01
  • I don't know what tools you have for examining process internals on Windows, but I'd definitely look into that funky 4kB thing. Is the transfer going over a network connection? If both SSDs are on the same physical machine, there's no need for that, and Git should be operating locally as long as you're using a file-oriented URL. Of course if/when you need to transfer from one machine to another the same problem is likely to recur. – torek Aug 11 '22 at 08:19
  • No network involved, that'is the point. I can even go offline. The source URL is just the other .git directory (absolute path). In the meantime, I have reverted the change of core.bifFileThreshold and tried again after reboot with no other memory eaters. And it finally worked that way, after remote compression took a while. – PasterOfMuppets Aug 12 '22 at 18:09

0 Answers0