1

I am aware of this question.

I am considering using Protobuf 3 for a file format, due to its efficient encoding, explicit schema and wide support. However one part of the schema that is very inconvenient is that it doesn't allow required fields.

Google has its reasons for removing required fields in Protobuf 3. The problem they had is real (removing required fields is a breaking change), but their solution is nonsense.

Anyway my question is: Protobuf 3 allows you to add custom options for fields. Has anyone used that (or another method) to add unofficial support for required fields to Protobuf 3?

Timmmm
  • 88,195
  • 71
  • 364
  • 509

2 Answers2

0

Do you want to have some implicit validation for your file format by using Protobuf, i.e. do you want the parsing of a file to fail if the required field does not exist in the data? If so, you may use the wrapper types, these will trigger an error if you try to read their value from a non-existing field.

Jan-Gerd
  • 1,261
  • 8
  • 8
  • No, I want a way of actually annotating the `.proto` file to say "this field is required". Then ideally I could use a modified Protobuf library that supports it. Or at least use an external tool that supports it to validate that protobuf data includes those required fields. – Timmmm Dec 05 '19 at 13:42
  • Like _data for this field must be present in the file_? That's the default behaviour in Protobuf 3, since all fields always exist with their default value. Maybe you can give an example of such a field and your intended behaviour on reading and writing the file? – Jan-Gerd Dec 05 '19 at 13:45
  • No, the default is that all fields are optional. You may be confused because missing primitive fields are set to a default value. See [explanation here](https://developers.google.com/protocol-buffers/docs/proto3#default). – Timmmm Dec 05 '19 at 14:43
  • Setting and sending them is optional, but they will always be in your parsed object as there is no longer a difference between an unset field and a field that has been set to its default value. From the perspective of your application, all sets will always be set. That's the same guarantee that you got from Protobuf 2 by declaring all fields as required. – Jan-Gerd Dec 05 '19 at 14:53
  • It's a property of the parsers. I'm talking about the binary format. It is perfectly possible to write a Protobuf decoder that indicates whether or not primitive fields were present or defaulted. In any case you're completely ignoring message fields. – Timmmm Dec 05 '19 at 15:20
  • That's not possible because default values for scalar fields may or may not be serialized in the binary format. A valid parser is not allowed to discriminate between those cases. So if you use Protobuf 3 as your file format you get those guarantees.For message fields, the concept of existence still exists, and the wrapper types make it available for each scalar type. So if you use StringValue instead of string, you can write your validator easily. – Jan-Gerd Dec 05 '19 at 22:54
  • Ah that's awkward. :-/ – Timmmm Dec 06 '19 at 15:28
0

Dropbox's Rust library supports annotations to do this!

  • (gogoproto.nullable)=false Generates non-nullable fields types
  • (rust.nullable)=false Generates oneofs as non-nullable (fail on deserialization)
  • (rust.err_if_default_or_unknown)=true Generates enums as non-zeroable (fail on deserialization)
Timmmm
  • 88,195
  • 71
  • 364
  • 509