Git log output to XML, JSON, or YAML?

Question

This is a pretty simple question: as a Git newbie I was wondering if there's a way for me to output my git log to a file, preferably in some kind of serialized format like XML, JSON, or YAML. Any suggestions?

Answer 1

to output to a file:

git log > filename.log

To specify a format, like you want everything on one line

git log --pretty=oneline >filename.log

or you want it a format to be emailed via a program like sendmail

git log --pretty=email |email-sending-script.sh

to generate JSON, YAML or XML it looks like you need to do something like:

git log --pretty=format:"%h%x09%an%x09%ad%x09%s"

This gist (not mine) perfectly formats output in JSON: https://gist.github.com/1306223

See also:

• http://git-scm.com/book/en/Git-Basics-Viewing-the-Commit-History
• Git how to save a preset git log --format
• How to parse the output of git log

Answer 2

I did something like this to create a minimal web api / javascript widget that would show the last 5 commits in any repository.

If you are doing this from any sort of scripting language, you really want to generate your JSON with something other than " for your quote character, so that you can escape real quotes in commit messages. (You will have them sooner or later, and it's not nice for that to break things.)

So I ended up with the terrifying but unlikely delimiter ^@^ and this command-line.

var cmd = 'git log -n5 --branches=* --pretty=format:\'{%n^@^hash^@^:^@^%h^@^,%n^@^author^@^:^@^%an^@^,%n^@^date^@^:^@^%ad^@^,%n^@^email^@^:^@^%aE^@^,%n^@^message^@^:^@^%s^@^,%n^@^commitDate^@^:^@^%ai^@^,%n^@^age^@^:^@^%cr^@^},\'';

Then (in node.js) my http response body is constructed from stdout of the call to git log thusly:

var out = ("" + stdout).replace(/"/gm, '\\"').replace(/\^@\^/gm, '"');
if (out[out.length - 1] == ',') {
    out = out.substring (0, out.length - 1);
}

and the result is nice JSON that doesn't break with quotes.

Answer 3

This script wraps git log and produces JSON output: https://github.com/paulrademacher/gitjson

Answer 4

I wrote this in Powershell to get git logdata and save it as json or other format:

$header = @("commit","tree","parent","refs","subject","body","author","commiter") 
[string] $prettyGitLogDump= (git log MyCoolSite.Web-1.4.0.002..HEAD --pretty=format:'%H|%T|%P|%D|%s|%b|%an|%cn;') 
$gldata = foreach ($commit in $prettyGitLogDump.Replace("; ",';') -split  ";", 0, "multiline") {
          $prop = $commit -split "\|"
          $hash = [ordered]@{}
          for ($i=0;$i -lt $header.count;$i++) {$hash.add($header[$i],$prop[$i])} 
          [pscustomobject]$hash
}
$gldata |  ConvertTo-Json | Set-Content -Path "GitLog.json" 

The headernames:

"commit","tree","parent","refs","subject","body","author","commiter"

have to be in sync with datafields :

--pretty=format:'%H|%T|%P|%D|%s|%b|%an|%cn;'

See prettyformat docs.
I choose pipe | as a separator. I am taking a risc that it is not used in the commit message. I used semicolon ; as a sep for every commit. I should of course chosen something else. You could try to code some clever regular expression to match and check if your separators are used in the commit message. Or you could code more complex regularexpression to match split point or code a powershell scriptblock to define the split.

The hardest line in the code to figure out was.

prettyGitLogDump.Replace("; ",';') -split ";", 0, "multiline"

You have to set option multiline becuase there can be CR/LF in the messages and then split stops - you can only set multiline if nr of split is given. Therefore second paramvalue 0 which means all.

(The Replace("; ",';') is just a hack I get a space after the first commit. So I remove space after commit separator. Probably there is a better solution.)

Anyway i think this could be a workable solution for windows users or powershells fans that want the log from git to see who made the commit and why.

Answer 5

git log has no provisions to escape quotes, backslashes, newlines etc., which makes robust direct JSON output impossible (unless you limit yourself to subset of fields with predictable content e.g. hash & dates).

However, the %w([width[,indent1[,indent2]]]) specifier can indent lines, making it possible to emit robust YAML with arbitrary fields! The params are same as git shortlog -w:

Linewrap the output by wrapping each line at width. The first line of each entry is indented by indent1 spaces, and the second and subsequent lines are indented by indent2 spaces.
width, indent1, and indent2 default to 76, 6 and 9 respectively.
If width is 0 (zero) then indent the lines of the output without wrapping them.

The YAML syntax that guarantees predictable parsing of indented text (with no other quoting needed!) is YAML block literal, not folded, with explicit number of spaces to strip e.g. |-2.

• The - option strips trailing newlines, which technically is lossy — it loses distinction between 1 vs. 4 trailing newlines, which are possible to produce with git commit --cleanup=verbatim — so in theory you might want |+2 for multiline fields. Personally, I prefer treating one-line commit messages as strings without newline characters.

Example:

git log --pretty=format:'- hash: %H
  author_date: %aI
  committer_date: %cI
  author: |-2
    %w(0,0,4)%an <%ae>%w(0,0,0)
  committer: |-2
    %w(0,0,4)%cn <%ae>%w(0,0,0)
  message: |-2
    %w(0,0,4)%B%w(0,0,0)'

%w affects later % placeholders but also literal lines so resetting the indent with %w(0,0,0) is necessary, otherwise text like committer: will also get indented.

output fragment (on reconstructed unix history):

- hash: 0d54a08ec260f3d554db334092068680cdaff26a
  author_date: 1972-11-21T14:35:16-05:00
  committer_date: 1972-11-21T14:35:16-05:00
  author: |-2
    Ken Thompson <ken@research.uucp>
  committer: |-2
    Ken Thompson <ken@research.uucp>
  message: |-2
    Research V2 development
    Work on file cmd/df.s
    
    Co-Authored-By: Dennis Ritchie <dmr@research.uucp>
    Synthesized-from: v2

- hash: 4bc99f57d668fda3158d955c972b09c89c3725bd
  author_date: 1972-07-22T17:42:11-05:00
  committer_date: 1972-07-22T17:42:11-05:00
  author: |-2
    Dennis Ritchie <dmr@research.uucp>
  committer: |-2
    Dennis Ritchie <dmr@research.uucp>
  message: |-2
    Research V2 development
    Work on file c/nc0/c00.c
    
    Synthesized-from: v2

Note I didn't put quotes around the dates, to show off parsing as native YAML timestamp type. Support may vary (especially since YAML v1.2 spec delegated things like timestamps and booleans to app-dependent schema?), so it may be pragmatic to make them "strings"...

To convert YAML to JSON, you can pipe through yq or similar tools.
Or a short in-place script e.g.

| ruby -e '
require "yaml"
require "json"
# to parse timestamps
require "date"
data = YAML.safe_load(STDIN.read(), permitted_classes: [Date, Time])
puts(JSON.pretty_generate(data))
'

Answer 6

Behold https://github.com/dreamyguy/gitlogg, the last git-log => JSON parser you will ever need!

Some of Gitlogg's features are:

• Parse the git log of multiple repositories into one JSON file.
• Introduced repository key/value.
• Introduced files changed, insertions and deletions keys/values.
• Introduced impact key/value, which represents the cumulative changes for the commit (insertions - deletions).
• Sanitise double quotes " by converting them to single quotes ' on all values that allow or are created by user input, like subject.
• Nearly all the pretty=format: placeholders are available.
• Easily include / exclude which keys/values will be parsed to JSON by commenting out/uncommenting the available ones.
• Easy to read code that's thoroughly commented.
• Script execution feedback on console.
• Error handling (since path to repositories needs to be set correctly).

Success, the JSON was parsed and saved.

Error 001: path to repositories does not exist.

Error 002: path to repositories exists, but is empty.