Canonical serialization of object

Question

I am looking to transfer an object securely between two servers transitively by use of a 3rd party.

Both servers and the 3rd party know the structure of the object, and it is up to the 3rd party to format the object (it may be json, xml, form-encoding, etc.).

class MyObject
{
    public int Id { get; set; }
    public string Name { get; set; }
    public int Age { get; set; }
}

There are many possible representations of this object which will cause problems if I try calculate the HMAC of it.

The following two representations are equivalent from the objects perspective, however, will produce completely different HMAC values:

JSON

{"Id":12345,Name:"Steve McQueen",Age:52}

JSON (but in different order

{Age:52,"Id":12345,Name:"Steve McQueen"}

Form Encoding

Age=52&Name=Steve%20McQueen&Id=12345

Is there any serialization built into .NET that takes the order of the fields into consideration?

I was thinking of using BinaryFormatter, however, I didn't see any guarantee that if you formatted the same object twice that it would result in the same binary output. The same can be said for the JavaScriptSerializer, or any other serializer, presumably because order is not relevant to their intended function (which is for serialization, not verification).

When and where do you want to calculate HMAC? You can calculate HMAC of actual object, like you would implement hashcode. like `GetHmac() { return Hmac(Id+Name+Age); }` — Andrey, Apr 02 '14 at 19:14
It would be created on one server, and validated on the other. The client can choose to use a different encoding when contacting the 2nd server then what was used for the 1st. — Matthew, Apr 02 '14 at 19:17
Possible solution? http://stackoverflow.com/questions/3330989/order-of-serialized-fields-using-json-net#answer-11309106 — Caramiriel, Apr 02 '14 at 19:33
@Matthew yes, so? you will calculate hmac, include it into the object, JSON or whatever serialize it. Deserialize it on the other server calc hmac the same way, and compare with the one included. — Andrey, Apr 03 '14 at 10:08

score 0 · Answer 1 · answered Jul 16 '14 at 15:22

0

Yes, there are Data Contracts!
Specifically, the DataMemberAttribute has an Order property:
"Gets or sets the order of serialization and deserialization of a member."

answered Jul 16 '14 at 15:22

Task

3,668
1
21
32

score -3 · Accepted Answer · edited Oct 23 '21 at 16:08

My solution to this was to just serialize it (to say json) and then just hmac the serialized data.

The hmac will only work on the json that was provided.

Trying to get 2 systems to serialize exactly the same is not realistic.

Doing the HMAC on the serialized data guarantees that there will be no way the HMAC could be calculated wrong, HMAC'ing an object in memory will always be contingent on things specific to the operating environment (such as NodeJS vs .NET vs Go) and internal representations (.NET decimal/float/double vs NodeJS number).

Futhermore, even if you happen to use the same language and framework, the receiver may not use the same serializer (Newtonsoft vs DataContracts vs JsonSerializer), and to get it 100% consistent is not sustainable.

Instead the solution is to perform the HMAC on the resulting JSON string, as performing an HMAC on a UTF-8 string is consistent between all operating environments as the deserialization is irrelevant.

Canonical serialization of object

2 Answers2