Why does Hyperledger fabric use protobuf instead of JSON?

Question

What are the advantages that protobuf provides which JSON cannot?

"protobuff is faster than JSON" - What does this mean?

Note: I am aware of the differences between protobuf and JSON.

Answer 1

Fabric nodes use gRPC for communication and as such, it uses protobuf data serialization.

In Fabric, there is no need for the data sent over the wire to be human readable, as the data is anyway being constructed without human intervention.

You usually use JSON when you have payloads that are either expected to be constructed by users such as a REST API that is simple enough for users to use with curl/wget/scripts or when you want to be be able to debug the data sent over the wire.

As @jpa pointed out in his comment, sending JSON has its drawbacks such as sending the data scheme itself with every message along with the actual data and it is inefficient compared to protobuf which only sends the data only and the data scheme is not sent over the wire.

However there is also a drawback in using protobuf, and that is - that protobuf serialization is not deterministic.

In a system where there are signature checks everywhere such as Fabric, if you send a signature over a message, the message must be sent as a blob of bytes.

This makes the Fabric transaction structure extremely inefficient because it contains nested encapsulations of messages because each "layer" is represented in the layer above it as an opaque blob of bytes and to access an inner field of a transaction message you need to unmarshal the blob of bytes into the actual message object.

This makes Fabric transaction processing very slow, as observed in a paper a couple of years ago:

Fabric uses gRPC for communication between nodes in the network. To prepare data for transmission, Protocol Buffers are used for serialization. To be able to deal with application and software upgrades over time, Fabric’s block structure is highly layered, where each layer is marshaled and unmarshaled separately. This leads to a vast amount of memory allocated to convert byte arrays into data structures.

Had Fabric transaction messaging been based on a deterministic marshaling scheme such as ASN1, then you would've been able to send a transaction without intermediate opaque byte blobs because signature checks would've involved serializing the message via the deterministic ASN1 serialization to obtain the blob of bytes that Fabric messages carry.