git show/log without shell escape sequences, for use with python sh

Question

I'm using python's sh to script git commands. For example, I do things like

import sh
git = sh.git.bake(_cwd='/some/dir/')

project_hash = git('rev-parse', 'HEAD').stdout.strip()
project_branch = git('rev-parse', '--abbrev-ref', 'HEAD').stdout.strip()
project_date = git('log', '-1', '--pretty=format:%ci').stdout.strip()

and then I write the project_hash, project_branch and project_date into a database, etc.

The trouble is git sometimes adds shell escape sequences to its output. For example,

print(repr(project_hash))
print(repr(project_branch))
print(repr(project_date))

leads to

'e55595222076bd90b29e184b6ff6ad66ec8c3a03'
'master'
'\x1b[?1h\x1b=\r2012-03-26 01:07:40 -0500\x1b[m\r\n\r\x1b[K\x1b[?1l\x1b>'

The first two strings are not a problem, but the last one, the date, has escape sequences.

Is there any way I can get rid of these, e.g. asking git not to output any escape sequences?

I have tried the "--no-color" option with the git log command. That did not help.

I would also be happy to strip them out in python itself, but I don't know how. I tried s.encode('ascii') where s is the date string. That did not make a difference.

Print stdout in Python without shell escape sequences addresses the same issue. The recommendation there is to use python's subprocess rather than sh. E.g., I could do

project_date = subprocess.check_output(["git", "log", "-1", "--pretty=format:%ci"], cwd='/some/dir/')

and

print(repr(project_date))

gives

'2012-03-26 01:07:40 -0500'

That is what I want, of course. However, if it is possible I would prefer to stick with sh, and so would like to know if I can avoid the escape sequences using sh.

Any suggestions?

Answer 1

Those are not color sequences, those look like terminal initialization sequences. Specifically:

ESC [ ? 1 h ESC =

is the sequence to turn on function-key-mode and

ESC [ ? 1 l ESC >

is the sequence to turn it off again. This suggests that git log is running things through your pager. I'm not quite sure why; normally git suppresses use of the pager when the output is a pipe (as it is with subprocess.Popen() at least, and I would think with sh, although I have not used the sh module).

(Pause to consult documentation...)

Aha! Per sh module docs, by default, the output of an sh-module-run command goes through a pseudo-tty. This is fooling git into running your pager.

As a slightly dirty work-around, you can run git --no-pager log ... to suppress the use of the pager, even when running with sh. Or, you can try the _tty_out=False argument (again, I have not used the sh module, you will have to experiment a bit). Amusingly, one of the examples at the bottom of the sh module documentation is git!

Answer 2

It seems like sh does the right thing. In python 2.7, this:

import sh
git = sh.git.bake(_cwd='/tmp/gittest/')

project_hash = git('rev-parse', 'HEAD')
project_branch = git('rev-parse', '--abbrev-ref', 'HEAD')
project_date = git('log', '-1', '--pretty=format:%ci')

print(repr(project_hash).strip())
print(repr(project_branch).strip())
print(repr(project_date).strip())

gives me:

500ddad67203badced9a67170b42228ffa269f53
master
2013-11-22 00:05:59 +1100

If you really want to strip out escapes, use the decoder tools provided by python (Process escape sequences in a string in Python)