17

I'm pushing a code of around 200 MB into a repo. It is taking a lot of time. Is there anyway we can display the progress bar, so that I can know how much amount of code is pushed into the repo?

Anish Shah
  • 7,669
  • 8
  • 29
  • 40
  • 4
    Pushing 200mb might also indicate that you're using git for something you shouldn't.. – Daenyth Mar 24 '14 at 19:11
  • 1
    What exactly do you mean by "code of around 200 MB"? Do you have a file that's 200 MB in size? Or do the sum of all changes that you've made in the commits that you're pushing actually equal 200 MB? –  Mar 25 '14 at 03:24
  • 1
    @Daenyth, there are perfectly fine cases when you need to push the whole repo - for example when migrating existing repository to a new Git hosted system - Github, Bitbucket, VSTS etc. Such scenarios are quite common. – Ivan Sep 09 '16 at 07:49
  • 1
    @Daenyth, as a developer - yes, less frequent, as a DevOps \ CD specialist I do that more than actual commits :) So, keep open mind! – Ivan Sep 09 '16 at 18:22
  • 3
    The purpose of the question isn't to be close-minded and assume what the OP is trying to do -- he's seeking an answer to a question that's answerable. Best to stay on-topic. – dylanh724 Apr 25 '18 at 10:06
  • I don't know why, but git push -u origin master did it... – Gabriel Pellegrino Feb 03 '22 at 15:22

2 Answers2

11

git push --progress will be more precise with Git 2.10 (Q3 2016)

See commit e376f17 from Jeff King (peff)

The index-pack command has two progress meters:

  • one for "receiving objects", and
  • one for "resolving deltas".

You get neither by default, or both with "-v".

But for a push through receive-pack, we would want only the "resolving deltas" phase, not the "receiving objects" progress.
There are two reasons for this:

  • One is simply that existing clients are already printing "writing objects" progress at the same time.
    Arguably "receiving" from the far end is more useful, because it tells you what has actually gotten there, as opposed to what might be stuck in a buffer somewhere between the client and server. But that would require a protocol extension to tell clients not to print their progress. Possible, but complexity for little gain.

  • The second reason is much more important.
    In a full-duplex connection like git-over-ssh, we can print progress while the pack is incoming, and it will immediately get to the client.
    But for a half-duplex connection like git-over-http, we should not say anything until we have received the full request.
    Anything we write is subject to being stuck in a buffer by the webserver. Worse, we can end up in a deadlock if that buffer fills up.

So our best bet is to avoid writing anything that isn't a small fixed size until we've received the full pack.


Update September 2016: Git 2.10 is there, and you can see an example of this progress meter at the GitHub blog post "Git 2.10 has been released":

https://cloud.githubusercontent.com/assets/3477155/18064845/34bb6956-6dff-11e6-8e3e-8a92c0bddaef.gif


Update Git 2.11 (Q4 2016)

Now, an incoming "git push" that attempts to push too many bytes can now be rejected by setting a new configuration variable at the receiving end.

See commit c08db5a, commit 411481b (24 Aug 2016) by Jeff King (peff).
See commit 5ad2186 (24 Aug 2016) by Christian Couder (chriscool).
(Merged by Junio C Hamano -- gitster -- in commit da3b6f0, 09 Sep 2016)

receive-pack: allow a maximum input size to be specified

Receive-pack feeds its input to either index-pack or unpack-objects, which will happily accept as many bytes as a sender is willing to provide.
Let's allow an arbitrary cutoff point where we will stop writing bytes to disk.

git config doc now includes:

receive.maxInputSize

If the size of the incoming pack stream is larger than this limit, then git-receive-pack will error out, instead of accepting the pack file.
If not set or set to 0, then the size is unlimited.


With Git 2.22, the progress is better managed:

See commit 545dc34, commit 9f1fd84 (12 Apr 2019), and commit d53ba84, commit 9219d12 (05 Apr 2019) by SZEDER Gábor (szeder).
(Merged by Junio C Hamano -- gitster -- in commit 425e51e, 25 Apr 2019)

See commit 1aed1a5 (19 May 2019) by SZEDER Gábor (szeder).
(Merged by Junio C Hamano -- gitster -- in commit fa03d9c, 30 May 2019)

progress: break too long progress bar lines

Some of the recently added progress indicators have quite long titles, which might be even longer when translated to some languages, and when they are shown while operating on bigger repositories, then the progress bar grows longer than the default 80 column terminal width.

When the progress bar exceeds the width of the terminal it gets line-wrapped, and after that the CR at the end doesn't return to the beginning of the progress bar, but to the first column of its last line.
Consequently, the first line of the previously shown progress bar is not overwritten by the next, and we end up with a bunch of truncated progress bar lines scrolling past:

$ LANG=es_ES.UTF-8 git commit-graph write
  Encontrando commits para commit graph entre los objetos empaquetados:   2% (1599
  Encontrando commits para commit graph entre los objetos empaquetados:   3% (1975
  Encontrando commits para commit graph entre los objetos empaquetados:   4% (2633
  Encontrando commits para commit graph entre los objetos empaquetados:   5% (3292
  [...]

Prevent this by breaking progress bars after the title once they exceed the width of the terminal, so the counter and optional percentage and throughput, i.e. all changing parts, are on the last line.
Subsequent updates will from then on only refresh the changing parts, but not the title, and it will look like this:

$ LANG=es_ES.UTF-8 ~/src/git/git commit-graph write
  Encontrando commits para commit graph entre los objetos empaquetados:
  100% (6584502/6584502), listo.
  Calculando números de generación de commit graph: 100% (824705/824705), listo.
  Escribiendo commit graph en 4 pasos: 100% (3298820/3298820), listo.

Git 2.23 (Q3 2019) fixes the progress display of a rebase:

See commit 5b12e31, commit cd1096b, commit 077b979, commit c9749b3 (24 Jun 2019), and commit d7d9088 (27 Jun 2019) by SZEDER Gábor (szeder).
(Merged by Junio C Hamano -- gitster -- in commit 6624e07, 09 Jul 2019)

rebase: fix garbled progress display with '-x'

When running a command with the 'exec' instruction during an interactive rebase session, or for a range of commits using 'git rebase -x', the output can be a bit garbled when the name of the command is short enough:

 $ git rebase -x true HEAD~5
 Executing: true
 Executing: true
 Executing: true
 Executing: true
 Executing: true)
 Successfully rebased and updated refs/heads/master.

Note the ')' at the end of the last line.
It gets more garbled as the range of commits increases:

 $ git rebase -x true HEAD~50
 Executing: true)
 [ repeated 3 more times ]
 Executing: true0)
 [ repeated 44 more times ]
 Executing: true00)
 Successfully rebased and updated refs/heads/master.

Those extra numbers and ')' are remnants of the previously displayed "Rebasing (N/M)" progress lines that are usually completely overwritten by the "Executing: <cmd>" lines, unless 'cmd' is short and the "N/M" part is long.

Make sure that the previously displayed "Rebasing (N/M)" line is cleared by using the term_clear_line() helper function added in the previous patch.
Do so only when not being '--verbose', because in that case these "Rebasing (N/M)" lines are not printed as progress (i.e. as lines with '\r' at the end), but as "regular" output (with '\n' at the end).

A couple of other rebase commands print similar messages, e.g. "Stopped at <abbrev-oid>... <subject>" for the 'edit' or 'break' commands, or the "Successfully rebased and updated <full-ref>." at the very end.

These are so long that they practically always overwrite that "Rebasing (N/M)" progress line, but let's be prudent, and clear the last line before printing these, too.

In 't3420-rebase-autostash.sh', two helper functions prepare the expected output of four tests that check the full output of 'git rebase' and thus are affected by this change, so adjust their expectations to account for the new line clearing.

Note that this patch doesn't completely eliminate the possibility of similar garbled outputs, e.g. some error messages from rebase or the "Auto-merging <file>" message from within the depths of the merge machinery might not be long enough to completely cover the last "Rebasing (N/M)" line.

This patch doesn't do anything about them, because dealing with them individually would result in way too much churn, while having a catch-all term_clear_line() call in the common code path of pick_commits() would hide the "Rebasing (N/M)" line way too soon, and it would either flicker or be invisible.


However, Git 2.24 (Q4 2019) includes a regression fix for progress output

See commit 2bb74b5, commit bbf4756 (16 Sep 2019) by SZEDER Gábor (szeder).
(Merged by Junio C Hamano -- gitster -- in commit ef93bfb, 07 Oct 2019)

Test the progress display

'progress.c' has seen a few fixes recently (commit 545dc34 and commit 9f1fd84, both v2.22.0-rc0), and, unfortunately, some of those fixes required further fixes (commit 1aed1a5).

The progress display is, however, critically dependent on timing, because it's only updated once every second or, if the total is known in advance, every 1%, and there is the throughput rate as well. These make the progress display far too undeterministic for testing as-is.

Hence:

progress: avoid empty line when breaking the progress line

Since commit 545dc34 (progress: break too long progress bar lines, 2019-04-12, Git v2.22.0-rc0) when splitting a too long progress line, sometimes it looks as if a superfluous empty line were added between the title line and the counters.

To make sure that the previously displayed progress line is completely covered up when writing the new, shorter title line, we calculate how many characters need to be overwritten with spaces.
Alas, this calculation doesn't account for the newline character at the end of the new title line, and resulted in printing one more space than strictly necessary.
This extra space character doesn't matter, if the length of the previous progress line was shorter than the width of the terminal.
However, if the previous line matched the terminal width, then this extra space made the new line longer, effectively adding that empty line after the title line.

Fix this off-by-one to avoid that spurious empty line.


With Git 2.25 (Q1 2020), one kind of progress messages were always given during commit-graph generation, instead of following the "if it takes more than two seconds, show progress" pattern, which has been corrected.

See commit ecc0869, commit 44a4693 (25 Nov 2019) by Derrick Stolee (derrickstolee).
(Merged by Junio C Hamano -- gitster -- in commit 41dac79, 10 Dec 2019)

commit-graph: use start_delayed_progress()

Helped-by: Jeff King
Reported-by: ryenus
Signed-off-by: Derrick Stolee

When writing a commit-graph, we show progress along several commit walks. When we use start_delayed_progress(), the progress line will only appear if that step takes a decent amount of time.

However, one place was missed: computing generation numbers. This is normally a very fast operation as all commits have been parsed in a previous step. But, this is showing up for all users no matter how few commits are being added.

The tests that check for the progress output have already been updated to use GIT_PROGRESS_DELAY=0 to force the expected output.

That new environment variable comes from:

progress: create GIT_PROGRESS_DELAY

Helped-by: Jeff King
Signed-off-by: Derrick Stolee

The start_delayed_progress() method is a preferred way to show optional progress to users as it ignores steps that take less than two seconds.
However, this makes testing unreliable as tests expect to be very fast.

In addition, users may want to decrease or increase this time interval depending on their preferences for terminal noise.

Create the GIT_PROGRESS_DELAY environment variable to control the delay set during start_delayed_progress().
Set the value in some tests to guarantee their output remains consistent.

The git documentation now includes:

GIT_PROGRESS_DELAY:

A number controlling how many seconds to delay before showing optional progress indicators.
Defaults to 2.


With Git 2.28 (Q3 2020), commit-graph can show progress of finding reachable commits.

See commit 2f00c35, commit 1f1304d, commit 0ec2d0f, commit 5b6653e, commit 630cd51, commit d335ce8 (13 May 2020), commit fa8953c (18 May 2020), and commit 1fe1084 (05 May 2020) by Taylor Blau (ttaylorr).
(Merged by Junio C Hamano -- gitster -- in commit dc57a9b, 09 Jun 2020)

commit-graph.c: show progress of finding reachable commits

Signed-off-by: Taylor Blau

When 'git commit-graph write --reachable ' is invoked, the commit-graph machinery calls 'for_each_ref()' to discover the set of reachable commits.

Right now the 'add_ref_to_set' callback is not doing anything other than adding an OID to the set of known-reachable OIDs. In a subsequent commit, 'add_ref_to_set' will presumptively peel references. This operation should be fast for repositories with an up-to-date '$GIT_DIR/packed-refs', but may be slow in the general case.

So that it doesn't appear that 'git commit-graph write' is idling with '--reachable' in the slow case, add a progress meter to provide some output in the meantime.

In general, we don't expect a progress meter to appear at all, since peeling references with a 'packed-refs' file is quick.
If it's slow and we do show a progress meter, the subsequent 'fill_oids_from_commits()' will be fast, since all of the calls to 'lookup_commit_reference_gently()' will be no-ops.

Both progress meters are delayed, so it is unlikely that more than one will appear. In either case, this intermediate state will go away in a handful of patches, at which point there will be at most one progress meter.


With Git 2.28 (Q3 2020), the code to produce progress output from "git commit-graph --write" had a few breakages, which have been fixed in the same 2.28 release.

See commit 150cd3b, commit 6f9d5f2 (09 Jul 2020) by SZEDER Gábor (szeder).
(Merged by Junio C Hamano -- gitster -- in commit 12f5eb9, 15 Jul 2020)

commit-graph: fix progress of reachable commits

Signed-off-by: SZEDER Gábor

To display a progress line while iterating over all refs, d335ce8f24 ("commit-graph.c: show progress of finding reachable commits", 2020-05-13, Git v2.28.0-rc0 -- merge listed in batch #2) should have added a pair of start_delayed_progress() and stop_progress() calls around a for_each_ref() invocation.

Alas, the stop_progress() call ended up at the wrong place, after write_commit_graph(), which does all the commit-graph computation and writing, and has several progress lines of its own.

Consequently, that new

Collecting referenced commits: 123  

progress line is overwritten by the first progress line shown by write_commit_graph(), and its final "done" line is shown last, after everything is finished:

Expanding reachable commits in commit graph: 344786, done.
Computing commit changed paths Bloom filters: 100% (344786/344786), done.
Collecting referenced commits: 154, done.  

Move that stop_progress() call to the right place.

While at it, drop the unnecessary 'if (data.progress)' condition protecting the stop_progress() call, because that function is prepared to handle a NULL progress struct.

See more at commit 862aead (10 Jul 2020) by SZEDER Gábor (szeder).
(Merged by Junio C Hamano -- gitster -- in commit d1ae8ba, 15 Jul 2020)

commit-graph: fix "Collecting commits from input" progress line

Signed-off-by: SZEDER Gábor
Reviewed-by: Derrick Stolee

VonC
  • 1,262,500
  • 529
  • 4,410
  • 5,250
7

It's not a progress "bar", but git push already reports progress by default when it's run from a terminal. From the official Linux kernel git documentation for git push:

--progress

Progress status is reported on the standard error stream by default when it is attached to a terminal, unless -q is specified. This flag forces progress status even if the standard error stream is not directed to a terminal.

The fact that you're trying to push 200 MB at once suggests that you might be doing something sub-optimally with git.