1

I'm using Protobuf 3 along with gRPC in distributed environment ("microservices").

Due to lack of supporting not-set/missing values in Protobuf 3 I got the following issue related to contract additivity.

Imagine I have Service A and couple of consumer services B and C owned by Team B and Team C.

If I add a field, say, boolean value to contract of Service A, at the first it will have default value which will be written, say, to database as is.

Then, Team B updates their service to talk using updated contract and passes 'true' as the field value. Then, Team C still uses old contract and calls the same service - value gets replaced to false. But Team C didn't mean it, moreover they weren't aware about that field at all.

Thus, Service A cannot extend contract at all because consumers that didn't get updated for various reasons yet are able to harm data and the Service A can do nothing about it.

In Thrift such things are done just by single check (.isSet()).

There are dirty workarounds like wrapping primitives into objects but it forces to use library-implementation-specific checks-by-reference (at least in java) which seems to be rather poor hack than robust solution. Also, eventually, I have to wrap everything in wrappers, which as you imagine is not great solution as well.

What are best practices you use to manage such situations in Protobuf 3 in 2017? How do you manage/coordinate contract updates between teams/services? Thanks

Note: this question is not exactly about how to implement absence of detection for not-set/missing values, but rather about how to live with that and follow Protobuf 3 philosophy.

Community
  • 1
  • 1
idntfy_m
  • 133
  • 1
  • 6

2 Answers2

2

I think the problem here is that trying to check for field presence this way is not really an idiomatic use of protocol buffers (not even in proto2). It sounds like you are trying to evolve your schema by adding new fields but not reading those new fields unless you're sure they came from an updated client. The idiomatic way is to do this instead: just make sure the defaults for the new fields are reasonable and maintain compatible behavior if they're not explicitly set. Then don't try to check for presence--just read the fields and older clients will get good default behavior.

To give you an example, let's say you're adding a new feature that can be enabled or disabled. The right way to do this would be to add a bool field in your request message called enable_new_feature. Since older clients don't know about this field, their requests will have this default to false and so they get the old behavior they're expecting. Adding a disable_new_feature field instead would probably be the wrong way to do it because then you would indeed break older clients by enabling something they didn't want.

Adam Cozzette
  • 1,396
  • 8
  • 9
  • Thanks, the most ideologically correct way of dealing with this I've seen so far. This would mean that whenever I don't agree with defaults - I have to send _a feature switcher_ additionally to those fields. – idntfy_m Dec 09 '16 at 20:12
  • Then, if I ever need to enforce new clients to follow new rules I have ability to enforce contract by requiring a feature switcher to be always true, which will make old clients to update. The thing here is though it is not exactly clear when I can get rid of those feature flags, because like any other feature flags, those ones have to have limited lifetime as well, otherwise there will be too much burden to support them. ...Which is essentially the same situation as with having `.isSet()` flags, but more explicit for consumer. Am I heading in the right direction? – idntfy_m Dec 09 '16 at 20:12
  • I think that's the right direction. I would say just delete the temporary feature flag as soon as all your code has been redeployed and no longer depends on it. – Adam Cozzette Dec 12 '16 at 16:05
1

Using oneof looks like a better/cleaner alternative to wrappers. See this answer to a similar question: https://stackoverflow.com/a/40552570/618259

Community
  • 1
  • 1
alavrik
  • 2,151
  • 12
  • 16
  • Thanks, you're comment made me to realize that I need to formulate the question better. Updated description, thank you. (Yep I know about this workaround, but this is not exactly what I'm asking) – idntfy_m Dec 07 '16 at 19:16