This video by John Williams gives an overview of what data goes into the calculation of a Git commit hash. Here's a screenshot from the video:
Reimplementing the commit hash without Git
To get a deeper understanding of this aspect of Git, I reimplemented the steps that produce a Git commit hash in Rust, without using Git. It currently works for getting the hash when committing a single file. The answers here were helpful in achieving this, thanks.
The source code of this answer is available here. Execute it with cargo run.
These are the individual pieces of data we need to compute to arrive at a Git commit hash:
- The object ID of the file, which involves hashing the file contents with SHA-1. In Git,
hash-object provides this ID.
- The object entries that go into the tree object. In Git, you can get an idea of those entries with
ls-tree, but their format in the tree object is slightly different: [mode] [file name]\0[object ID]
- The hash of the tree object which has the form:
tree [size of object entries]\0[object entries]. In Git, get the tree hash with: git cat-file commit HEAD | head -n1
- The commit hash by hashing the data you see with
cat-file. This includes the tree object hash and commit information like author, time, commit message, and the parent commit hash if it's not the first commit.
Each step depends on the previous one. Let's start with the first.
Get the object ID of the file
The first step is to reimplement Git's hash-object, as in git hash-object your_file.
We create the object hash from our file by concatenating and hashing these data:
- The string "blob " at the beginning (mind the trailing space), followed by
- the number of bytes in the file, followed by
- a null byte, expressed with
\0 in printf and Rust, followed by
- the file content.
In Bash:
file_name="your_file";
printf "blob $(wc -c < "$file_name")\0$(cat "$file_name")" | sha1sum
In Rust:
// Get the object ID
fn git_hash_object(file_content: &[u8]) -> Vec<u8> {
let file_size = file_content.len().to_string();
let hash_input = [
"blob ".as_bytes(),
file_size.as_bytes(),
b"\0",
file_content,
]
.concat();
to_sha1(&hash_input)
}
I'm using crate sha1 version 0.10.5 in to_sha1:
fn to_sha1(hash_me: &[u8]) -> Vec<u8> {
use sha1::{Digest, Sha1};
let mut hasher = Sha1::new();
hasher.update(hash_me);
hasher.finalize().to_vec()
}
Get the object entry of the file
Object entries are part of Git's tree object. Tree objects represent files and directories.
Object entries for files have this form: [mode] [file name]\0[object ID]
We assume the file is a regular, non-executable file, which translates to mode 100644 in Git. See this for more on modes.
This Rust function takes the result of the previous function git_hash_object as the parameter object_id:
fn object_entry(file_name: &str, object_id: &[u8]) -> Vec<u8> {
// It's a regular, non-executable file
let mode = "100644";
// [mode] [file name]\0[object ID]
let object_entry = [
mode.as_bytes(),
b" ",
file_name.as_bytes(),
b"\0",
object_id,
]
.concat();
object_entry
}
I tried to write the equivalent of object_entry in Bash, but Bash variables cannot contain null bytes. There are probably ways around that limitation, but I decided for now that if I can't have variables in Bash, the code would get quite difficult to understand. Edits providing a readable Bash equivalent are welcome.
Get the tree object hash
As mentioned above, tree objects represent files and directories in Git. You can see the hash of your tree object by running, for example, git cat-file commit HEAD | head -n1.
The tree object has this form: tree [size of object entries]\0[object entries]
In our case we only have a single object_entry, calculated in the previous step:
fn tree_object_hash(object_entry: &[u8]) -> String {
let object_entry_size = object_entry.len().to_string();
let tree_object = [
"tree ".as_bytes(),
object_entry_size.as_bytes(),
b"\0",
object_entry,
]
.concat();
to_hex_str(&to_sha1(&tree_object))
}
Where to_hex_str is defined as:
// Converts bytes to their hexadecimal representation.
fn to_hex_str(bytes: &[u8]) -> String {
bytes.iter().map(|byte| format!("{byte:02x}")).collect()
}
In a Git repo, you can look at the contents of the tree object with ls-tree. For example, running git ls-tree HEAD will produce lines like these:
100644 blob b8c0d74ef5ccd3dab583add7b3f5367efe4bf823 your_file
While those lines contain the data of an object entry (the mode, the object ID, and the file name), they are in a different order and include a tab character as well as the string "blob" which is input to the object ID, not the object entry. Object entries have this form: [mode] [file name]\0[object ID]
Get the commit hash
The last step creates the commit hash.
The data we hash using SHA-1 includes:
- Tree object hash from the previous step.
- Hash of the parent commit if the commit is not the very first one in the repo.
- Author name and authoring date.
- Committer name and committing date.
- Commit message.
You can see all of that data with git cat-file commit HEAD, for example:
tree a76b2df314b47956268b0c39c88a3b2365fb87eb
parent 9881a96ab93a3493c4f5002f17b4a1ba3308b58b
author Matthias Braun <m.braun@example.com> 1625338354 +0200
committer Matthias Braun <m.braun@example.com> 1625338354 +0200
Second commit (that's the commit message)
You might have guessed that 1625338354 is a timestamp. In this case it's the number of seconds since the Unix epoch. You can convert from the date and time format of git log, such as "Sat Jul 3 20:52:34 2021", to Unix epoch seconds with date:
date --date='Sat Jul 3 20:52:34 2021' +"%s"
The time zone is denoted as +0200 in this example.
Based on the output of cat-file, you can create the Git commit hash using this Bash command (which uses git cat-file, so it's no reimplementation):
cat_file_output=$(git cat-file commit HEAD);
printf "commit $(wc -c <<< "$cat_file_output")\0$cat_file_output\n" | sha1sum
The Bash command illustrates that—similar to the steps before—what we hash is:
- A leading string, "commit " in this step, followed by
- the size of a bunch of data. Here it's the output of
cat-file which is detailed above. Followed by
- a null byte, followed by
- the data itself (output of
cat-file) with a line break at the end.
In case you kept score: Creating a Git commit hash involves using SHA-1 at least three times.
Below is the Rust function for creating the Git commit hash. It uses the tree_object_hash produced in the previous step and a struct CommitMetaData which contains the rest of the data you see when calling git cat-file commit HEAD. The function also takes care of whether the commit has a parent commit or not.
fn commit_hash(commit: &CommitMetaData, tree_object_hash: &str) -> Vec<u8> {
let author_line = format!(
"{} {}",
commit.author_name_and_email, commit.author_timestamp_and_timezone
);
let committer_line = format!(
"{} {}",
commit.committer_name_and_email, commit.committer_timestamp_and_timezone
);
// If it's the first commit, which has no parent,
// the line starting with "parent" is omitted
let parent_commit_line = match commit.parent_commit_hash {
Some(parent_commit_hash) => format!("\nparent {parent_commit_hash}"),
None => "".to_string(),
};
let git_cat_file_str = format!(
"tree {}{}\nauthor {}\ncommitter {}\n\n{}\n",
tree_object_hash, parent_commit_line, author_line, committer_line, commit.commit_message
);
let git_cat_file_len = git_cat_file_str.len().to_string();
let commit_object = [
"commit ".as_bytes(),
git_cat_file_len.as_bytes(),
b"\0",
git_cat_file_str.as_bytes(),
]
.concat();
// Return the Git commit hash
to_sha1(&commit_object)
}
Here's CommitMetaData:
#[derive(Debug, Copy, Clone)]
pub struct CommitMetaData<'a> {
pub(crate) author_name_and_email: &'a str,
pub(crate) author_timestamp_and_timezone: &'a str,
pub(crate) committer_name_and_email: &'a str,
pub(crate) committer_timestamp_and_timezone: &'a str,
pub(crate) commit_message: &'a str,
// All commits after the first one have a parent commit
pub(crate) parent_commit_hash: Option<&'a str>,
}
This function creates CommitMetaData where author and committer info are identical, which will be convenient when we run the program later:
pub fn simple_commit<'a>(
author_name_and_email: &'a str,
author_timestamp_and_timezone: &'a str,
commit_message: &'a str,
parent_commit_hash: Option<&'a str>,
) -> CommitMetaData<'a> {
CommitMetaData {
author_name_and_email,
author_timestamp_and_timezone,
committer_name_and_email: author_name_and_email,
committer_timestamp_and_timezone: author_timestamp_and_timezone,
commit_message,
parent_commit_hash,
}
}
Putting it all together
As a summary and reminder, creating a Git commit hash consists of getting:
- The object ID of the file, which involves hashing the file contents with SHA-1. In Git,
hash-object provides this ID.
- The object entries that go into the tree object. In Git, you can get an idea of those entries with
ls-tree, but their format in the tree object is slightly different: [mode] [file name]\0[object ID]
- The hash of the tree object which has the form:
tree [size of object entries]\0[object entries]. In Git, get the tree hash with: git cat-file commit HEAD | head -n1
- The commit hash by hashing the data you see with
cat-file. This includes the tree object hash and commit information like author, time, commit message, and the parent commit hash if it's not the first commit.
In Rust:
pub fn get_commit_hash(
file_name: &str,
file_content: &[u8],
commit: &CommitMetaData
) -> String {
let file_object_id = git_hash_object(file_content);
let object_entry = object_entry(file_name, &file_object_id);
let tree_object_hash = tree_object_hash(&object_entry);
let commit_hash = commit_hash(commit, &tree_object_hash);
to_hex_str(&commit_hash)
}
With the functions above, you can create a file's Git commit hash in Rust, without Git:
use std::{fs, io};
fn main() -> io::Result<()> {
let file_name = "your_file";
let file_content = fs::read(file_name)?;
let first_commit = simple_commit(
"Firstname Lastname <test@example.com>",
// Timestamp calculated using: date --date='Wed Jun 23 18:02:18 2021' +"%s"
"1624464138 +0200",
"Message of first commit",
// No parent commit hash since this is the first commit
None,
);
let first_commit_hash = get_commit_hash(file_name, &file_content, &first_commit);
Ok(println!("Git commit hash: {first_commit_hash}"))
}
To create the hash of the second commit, you take the hash of the first commit and put it into the CommitMetaData of the second commit:
let second_commit = simple_commit(
"Firstname Lastname <test@example.com>",
"1625388354 +0200",
"Message of second commit",
// The first commit is the parent of the second commit
Some(first_commit_hash),
);
Apart from the other answers here and their links, these were some useful resources in creating my limited reimplementation:
- Reimplementation of
git hash-object in JavaScript.
- Format of a Git tree object, this is the next place I'd look if I wanted to make my reimplementation more complete: To work with commits involving more than one file.
