Python subprocess "git gc" stderr is not captured

Question

Here's my observed behavior:

>>> out = subprocess.check_output("git gc", shell=True)
Counting objects: 4869, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (1219/1219), done.
Writing objects: 100% (4869/4869), done.
Total 4869 (delta 3607), reused 4869 (delta 3607)

The operation output is printed in STDERR. I wanted to capture this one in a variable too so I sent stderr to STDOUT. But it does not capture it.

>>> out = subprocess.check_output("git gc", shell=True, stderr=subprocess.STDOUT)
>>> out
b''
>>> print(out)
b''

Any ideas/suggestions?

It seems that git gc is a special case where output redirection is not possible. Using typescript was recommended in the other question, but can someone with more knowledge explain that behavior? The script approach did not work as it did not exit the fork.

Possible duplicate of [Trying to redirect 'git gc' output](http://stackoverflow.com/questions/18002585/trying-to-redirect-git-gc-output) — dsh, Apr 28 '16 at 15:42

score 1 · Answer 1 · edited May 23 '17 at 11:52

git gc is really a special case, see here: Trying to redirect 'git gc' output
Try using git status to see the difference.

Update

The question was a nice challenge. A co-worker looked at the source code of GIT and found out that git gc only writes to stdout if the process is not started in background. After some searching I came across the pty library and inspired by https://stackoverflow.com/a/6953572/2776376 found a working solution.
The following code catches the stdout of git gc, although it has to be written to a file.

#!/usr/bin/env python

import os
import pty
import sys

log_name = 'git_log.txt'

def git_logger():
    (child_pid, fd) = pty.fork()
    if child_pid == 0:
        #parent process
        sys.stdout.flush()
        os.execlp("git", "git", "gc")
    else:
        #child process
        output = os.read(fd, 100)
        while True:
            try:
                output += os.read(fd, 1024)
            except:
                break
        with open(log_name, 'w') as log_file:
            log_file.write(output)

if __name__ == "__main__":
    git_logger()
    with open(log_name, 'r') as log_file:
        print(log_file.read())

score 0 · Answer 2 · edited May 23 '17 at 11:52

git gc may change its behavior depending whether its standard input/output/error streams are connected to a terminal or not (whether it thinks, it is run interactively or not). If you want the same output as if you were running it in the terminal; you could use a pseudo-tty (it seems there is no command-line option that would configure the desired output behavior for git gc: --quiet doesn't accept a parameter e.g., like it does with apt-get):

#!/usr/bin/env python
import pexpect # $ pip install pexpect

output, exitstatus = pexpect.run('git gc', withexitstatus=1)

Related questions with pty-based solutions:

Python subprocess "git gc" stderr is not captured

2 Answers2

Update