8

Google has removed unknown fields in proto3. I would love to hear the reasoning behind this choice. Also, if anyone has any way to replicate the proto2 behavior I would love to hear it.

If it matters, we are writing our code in Go.

As proto3 and grpc were developed in parallel, I wanted to reach out to the grpc community as well.

Source: Removal of unknown fields

Andriy Drozdyuk
  • 58,435
  • 50
  • 171
  • 272
Joshua
  • 317
  • 2
  • 6
  • 1
    It's not about who you're "reaching out to". You're asking a question about Protocol Buffers. gRPC is a different thing. Sure, they're related, but your question isn't *about* gRPC. – murgatroid99 Apr 08 '15 at 22:10
  • They don't seem to have said much. As a guess, maybe it cleans up the interface: gRPC [just fills your structs](http://godoc.org/google.golang.org/grpc#Stream) whereas proto [needs to give you a message to pass along](http://godoc.org/github.com/golang/protobuf/proto#Unmarshal) (and you need to actually pass it along to keep your app unknown-field-safe). They might have figured it safer to make callers and callees upgrade together, in the same way that, say, when a Go method adds a parameter its callers need to update as well. As a workaround, maybe you could use proto3 maps for new fields. – twotwotwo Apr 29 '15 at 21:07
  • 1
    Maybe provide a specific use-case for which you used "unknown" fields? – Andriy Drozdyuk May 12 '15 at 20:51
  • 2
    If a protobuf message is stored in a database and a client with an older version of the protobuf message type needs to make a modification without clobbering the fields written by clients with the current version of the protobuf. Updating all clients simultaneously to have the latest protobuf message version is not a possibility. Further, there are different client types that have different update schedules, so the change would need to be coordinated across multiple codebases. – Joshua May 13 '15 at 05:19
  • Sure, this makes release engineering a bit harder but it's certainly not impossible. It is sufficient to first release all components with the new proto definition built in and only in the next release start writing the new fields. – siimphh May 30 '15 at 10:55
  • Looks like this will be implemented officially now: https://github.com/google/protobuf/issues/272#issuecomment-286249933 https://docs.google.com/document/d/1KMRX-G91Aa-Y2FkEaHeeviLRRNblgIahbsk4wA14gRk/edit#heading=h.w8dtggryroj4 – Joshua Jun 23 '17 at 17:21

1 Answers1

2

Getting rid of field presence for primitives makes Protobuf more "natural" and efficient in many languages, since primitives in C/C++, Java, C#, and Go must be present. In such languages, if you want presence information you "box" the primitive by making it a pointer to a primitive.

Protobuf 3 gets rid of presence for primitives but still has it for messages. Thus, you can use the same "boxing" technique for Protobuf. Protobuf now has standard messages that box primitives.

Eric Anderson
  • 24,057
  • 5
  • 55
  • 76