Digest doesn't match when inserting to QLDB

Question

I'm trying to insert a record into a QLDB ledger using the AWS SDK in Go. I used the Python QLDB driver as an example and logged the final transaction hash produced there. This is used during the transaction commit to compare with the hash produced on QLDB's side to verify the transaction and allow it to commit, which the python driver does successfully.

There isn't a Go version of IonHash yet, though, so I've implemented up the StartTransaction, InsertInto, and CommitTransaction steps in Go and included a Python executable IonHash implementation to calculate the IonHashes used to compare the digest at the end.

// Go (pseudocode)
import "github.com/fernomac/ion-go" as ion
import python_hash_module as python

func (client qldbClient) StartTransaction(transactionID string) {
 // hash transactionID using python ionhash
}

func (client) InsertInto (statement string, params string) {
    // MarshalText using ion module in aws-sdk
    ionParam := ion.MarshalText(params)

    // hash statement using python executable
    client.statementHash = python.ion_hash(statement)

    // hash parameters using python executable (only one parameter)
    client.paramHash = python.ion_hash(ionParam)

    // dot paramHash with statement hash
    client.statementHash = client.statementHash.dot(client.paramHash)

    // dot statement hash with transactionhash - this transaction hash matches the python calculation!
    client.transactionHash = client.transactionHash.dot(statementHash)
}

func (client) Commit() {
    res, err := client.execute(statement) // compares calculated transaction hash with AWS calculated transaction hash
    if err != nil {
        log.Prinln(err)
}

The code fails during the commit step with the following error:

{
  Code_: "412",
  Message_: "Digests don't match"
}
2020/03/22 11:16:41 xxxx.go:xxx: BadRequestException: Digests don't match
{
  Code_: "412",
  Message_: "Digests don't match"
}

I don't understand why the digests don't match during the commit in go, when this implementation is producing the same digest as the python code, which does commit. Why does the python code not complain about digests not matching when it's producing the same commits as the go code? More importantly, how can I successfully insert to QLDB through Go (not the python or the node drivers?)

Answer 1

Not sure if this is still helpful, but Amazon released a preview version of a QLDB Go Driver recently (https://github.com/awslabs/amazon-qldb-driver-go).

It has Ion and Ion Hash as its dependencies, so this should make things much easier for you when working with QLDB.

Answer 2

The understanding that ion-hash-go is not available is correct.

A couple of things you might want to check while computing hash:

1. The hash value is seeded with the ion hash of transaction id. So Technically final hash of two transactions would never be the same.
2. As you note, The Hash value is updated with the QLDB ‘dot’ operator. The dot operator is the way QLDB merges hash values and is defined as the hash of the concatenation of the two hashes, ordered by the (signed, little-endian) byte-wise comparison between the two hashes. ref - https://github.com/awslabs/amazon-qldb-driver-python/blob/39fecdf2ed0521ae1d19f342a4cab38846b96c9a/pyqldb/util/qldb_hash.py#L99
3. The Hashing algorithm needs to be SHA 256.
4. While Hashing queries with parameters, the correct implementation is to take the ion hash of any statement (with any PartiQL literals that may be in there) and then the IonHash of each IonValue passed in as a parameter (combined with dot operator).

From what I can guess from your pseudo code is that you could be missing 1) and 2). More specifically I mean to say that the pseudo suggests that the hash is not seeded with transaction id and dot operator is not applied to the hashes in right order.

For more details on QLDB hash calculation, Refer Marc's answer - How to get/compute CommitDigest when committing a transaction in AWS QLDB?

Open to helping on this further.

Update:

Ion hash go is now available - https://github.com/amzn/ion-hash-go

Also go driver to interact with QLDB has been released - https://github.com/awslabs/amazon-qldb-driver-go

Both of them are currently beta.