Go Protobuf declarations and optional Fields in Go Struct (string pointers)

Question

I am running into a bit of a problem with Protoc and my existing struct that contains nullable string fields.

The struct I am trying to serialize for transmission contains a bunch of fields that are nullable in json (so we can distinguish between null, "" and a set value).

type Message struct {
  Path *string `json:"path"`
}

So if a user sends a empty json string {} the Path will be nil and not "", whereas {"path":""} is also valid and a different case from {"path": null}.

The proto3 declaration I came up with obviously looks like this (and is optional as required and optional got dropped from proto3:

syntax = "proto3";
message Message {
  string Path = 1;
}

After running Protoc I end up with a struct that looks like this and all values are string and no way to declare them as *string:

type Message struct {
  Path string `protobuf:"bytes,1,opt,name=Path,proto3" json:"Path,omitempty"`
}

Obviously I can't assign to this array from my existing struct. But even if I were to write the tedious mapping code with target.Path = *source.Path with the appropriate null pointer checks etc I'd loose the triple meaning of my source struct (nil, "", "value").

Any suggestions on how to proceed here or if there is an extension to the Go Protobuf to do this? Or how would one go about describing this proto declaration?

mattes · Accepted Answer · 2021-04-30T18:59:05.767

7

Proto3 returns the Zero Value even if a field isn't set. Currently there is no way to distinguish if a field has been set or not.

See Github issue #15.

Possible solutions:

Use proto2 instead of proto3.
Use gogoproto's nullable extension.
Use the google.protobuf.FieldMask extension, see Common Design Patterns from the Google API Design Guide: Partial Responses and Output Fields.

edited Apr 30 '21 at 18:59

answered Jul 15 '19 at 22:05

mattes

8,936
5
48
73

score 1 · Answer 2 · answered Mar 25 '21 at 07:42

1

https://stackoverflow.com/a/62566052 proto3 support optional with flag --experimental_allow_proto3_optional

answered Mar 25 '21 at 07:42

Jonyhy96

168
1
2
10

score 0 · Answer 3 · answered May 15 '20 at 07:46

In my case, I solved this problem with several packages:

My proto file looks like this:

syntax = "proto3";

import "google/protobuf/wrappers.proto";
import "github.com/gogo/protobuf/gogoproto/gogo.proto";

message Message {
  google.protobuf.StringValue path = 1 [(gogoproto.wktpointer) = true];
}

The command for generating the go code, which I used look like this:

protoc -I. -I%GOPATH%/src --gogofaster_out=plugins=grpc:. proto/*.proto

The generated go file looks like this:

type Message struct {
    Path *string `protobuf:"bytes,1,opt,name=path,json=path,proto3,wktptr" json:"path,omitempty"`
}

Go Protobuf declarations and optional Fields in Go Struct (string pointers)

3 Answers3