Check if a field has been set in protocol buffer 3

Question

I am migrating a java application from protocol buffers 2 to protocol buffer 3.

In proto 2 to check if a field is set you have hasfield() method for which an example Java code generated is:

public boolean hasText() {
  return ((bitField0_ & 0x00000004) == 0x00000004);
}

However in proto 3 there it has been removed. How do you check if a field has been set in proto 3?

AFAIK, that simply isn't tracked in the default proto3 implementations. Fields are either zero (and not sent) or are non-zero (and are sent), and *that's it*. — Marc Gravell, Aug 19 '18 at 15:21
Yeah, proto3 screwed up with their optimization that scalars with default value aren't serialized. Now, a reader can't distinguish between "field not specified" and "field specified but with default value." You can claw back this functionality by wrapping your scalar inside a oneof or inside a submessage with the attendant cruftiness that incurs in your preferred language. — jschultz410, Feb 21 '20 at 20:09

score 12 · Answer 1 · answered Aug 20 '18 at 05:45

One of the suggested approaches is given here:

# NOTE: As of proto3, HasField() only works for message fields, not for
#       singular (non-message) fields. First try to use HasField and
#       if it fails (with a ValueError) we manually consult the fields.
try:
    return message_pb.HasField(property_name)
except ValueError:
    all_fields = set([field.name for field in message_pb._fields])
    return property_name in all_fields

Also, from the same page:

In proto3, field presence for scalar fields simply doesn't exist. Your mental model for proto3 should be that it's a C++ or Go struct. For integers and strings, there is no such thing as being set or not, it always has a value. For submessages, it's a pointer to the submessage instance which can be NULL, that's why you can test presence for it.

Yeah, if you want presence / absence for scalars then you either need to wrap them in a oneof or inside a message (e.g. - google.protobuf.wrappers.proto). — jschultz410, Feb 21 '20 at 20:05

UsamaAmjad · Answer 2 · 2021-03-17T23:36:13.000

Use Proto Wrappers `hasField()`

I see some people are suggesting to use oneof to wrap your field in it but proto3 already provides built-in wrappers with hasField kind of methods for primitive data types which will allow you to check if the field has been set or not.

This is how you can declare it (see more Value types here):

syntax = "proto3";

import "google/protobuf/wrappers.proto";

message MyProtoMessage {
    google.protobuf.BoolValue enabled = 1;
    google.protobuf.StringValue name = 2;
    google.protobuf.Int32Value age = 3;
}

and this in Java code to check it:

MyProtoMessage myProtoMessage = getItFromSomewhere();
if (myProtoMessage.hasName()) {
    // field is set by client
    myProtoMessage.getName();
} else {
    // field is not set by the client
    // but you can still call `myProtoMessage.getName()` which will `return` default value ""
}

score 8 · Answer 3 · answered Mar 09 '21 at 06:12

8

Protobuf 3.15.0 now has the support for optional fields. You can use hasField methods again.

answered Mar 09 '21 at 06:12

Lahiru Chandima

22,324
22
103
179

1

Yes. Interestingly enough, this is basically just syntactic sugar for an implicit oneof that wraps the scalar. – jschultz410 Sep 13 '21 at 21:13

jschultz410 · Answer 4 · 2021-09-13T21:07:52.690

The best suggestion I've seen for this in proto3 is to wrap your field in a singleton oneof. This will allow you to check for presence / absence again, similar to proto2.

message blah
{
    oneof foo_ { sint32 foo = 1; }
}

In Python generated code this is extremely smooth as foo can be directly operated on as a scalar as if it wasn't inside a oneof.

Unfortunately, for Java I think support for oneof's is far uglier. Google also purposely removed the hasFoo() generated class function in proto3. So you would need to consult the getFooCase() of the oneof instead to check for presence or absence.

https://developers.google.com/protocol-buffers/docs/reference/java-generated#oneof-fields

Yes, I realize this means tons of oneof's and any attendant hassle they bring. On the plus side, there is no overhead on the wire.

The second best suggestion I've seen is to use a submessage to wrap your scalar because presence / absence of submessages is still supported. There are the Well Known Types (WKT) in google.protobuf.wrappers.proto. If you use those, then you may even get extra special treatment in your preferred language where the wrapped scalar can be easily operated on as if the containing submessage didn't exist (or so I've read, not entirely sure on this point myself).

score 3 · Answer 5 · answered Mar 22 '19 at 02:36

3

I think the recommended approach, which is not ideal because of design decisions made in proto3, is to check standard values. You cannot explicitly check whether a field is set or not. As access to msg._fields is not recommended as described here the only thing that is left is to check whether the field is set to its standard value:

if msg.textfield.isEmpty() {
    //assume textfield is not set
}

answered Mar 22 '19 at 02:36

Gigi

150
1
8

1

This can work. You can also wrap it in `oneof`. `oneof test_string { string name = 1; }`. That also has some pitfalls. – artless noise Oct 06 '19 at 22:54

Check if a field has been set in protocol buffer 3

5 Answers5

Use Proto Wrappers hasField()

Use Proto Wrappers `hasField()`