How to convert from Json to Protobuf?

Question

I'm new to using protobuf, and was wondering if there is a simple way to convert a json stream/string to a protobuf stream/string in Java?

For example,

protoString = convertToProto(jsonString)

I have a json string that I want to parse into a protobuf message. So, I want to first convert the json string to protobuf, and then call Message.parseFrom() on it.

You might want to take a look at [FlatBuffers](https://google.github.io/flatbuffers/) instead. — Gergely Kőrössy, Jul 15 '16 at 23:45
"is a simple way to convert a json stream/string to a protobuf stream/string? "Probably not. They work differently; protobuf fields aren't actually named in the serialized representation, but they are in a sequence. JSON is almost the opposite; fields are named, but the sequence generally doesn't matter. What you *can* do is take your generated protobuf classes and use Jackson mixins to annotate them and parse into them. — David Ehrmann, Jul 16 '16 at 00:56
Related - if it helps, here is some utility to view protos as json. It has both, a Java API, as well as an installable GUI tool. You can also check the source code to see "how it does that": https://github.com/Zabuzard/ProtoToJson — Zabuzard, Sep 22 '22 at 18:34
Protobuf 3 has a built-in [JSON mapping](https://developers.google.com/protocol-buffers/docs/proto3#json) — Florian Enner, Jan 02 '23 at 13:00

score 55 · Answer 1 · answered Jul 18 '16 at 16:06

55

With proto3 you can do this using JsonFormat. It parses directly from the JSON representation, so there is no need for separately calling MyMessage.parseFrom(...). Something like this should work:

JsonFormat.parser().merge(json_string, builder);

answered Jul 18 '16 at 16:06

Adam Cozzette

1,396
8
9

7

When a value is invalid (say proto is boolean, but json value is integer), is there a way to configure the parser to ignore that field or set that field to null, instead of throwing `com.google.protobuf.InvalidProtocolBufferException`? – the1plummie Jan 31 '17 at 03:32
This doesnt seem to work for my class - I get `com.google.protobuf.InvalidProtocolBufferException: JsonObject` - any suggestions? – Brad Parks Oct 25 '19 at 18:41

score 14 · Answer 2 · answered Oct 02 '19 at 09:23

//You can use this for converting your input json to a Struct / any other Protobuf Class    

import com.google.protobuf.Struct.Builder;
import com.google.protobuf.Struct;
import com.google.protobuf.util.JsonFormat;
import org.json.JSONObject;

JSONObject parameters = new JSONObject();

Builder structBuilder = Struct.newBuilder();
JsonFormat.parser().merge(parameters.toString(), structBuilder);

// Now use the structBuilder to pass below (I used it for Dialog Flow V2 Context Management)

While this code may answer the question, providing additional context regarding why and/or how this code answers the question improves its long-term value. — xiawi, Oct 02 '19 at 09:41

score 2 · Answer 3 · answered Jul 28 '21 at 00:06

Since someone asked about getting the exception "com.google.protobuf.InvalidProtocolBufferException: JsonObject" when following Adam's advice--I ran into the same issue. Turns out it was due to the google protobuf timestamps. They are being serialized as an object containing two fields "seconds" and "nanos", since this isn't production code, I just got around this by parsing the JSON using jackson, going through the JSON object recursively and changing every timestamp from an object to a string formatted as per RFC 3339, I then serialized it back out and used the protobuf JSON parser as Adam has shown. This fixed the issue. This is some throwaway code I wrote (in my case all timestamp fields contain the word "timestamp", this could be more robust, but I don't care):

public Map<String, Object> fixJsonTimestamps(Map<String, Object> inMap) {
    Map<String, Object> outMap = new HashMap<>();
    for(String key : inMap.keySet()) {
        Object val = inMap.get(key);
        if(val instanceof Map) {
            Map<String, Object> valMap = (Map<String, Object>)val;
            if(key.toLowerCase().contains("timestamp") && 
                    valMap.containsKey("seconds") && valMap.containsKey("nanos")) {
                if(valMap.get("seconds") != null) {
                    ZonedDateTime d = ZonedDateTime.ofInstant(Instant.ofEpochSecond((int)valMap.get("seconds")).plusNanos((int)valMap.get("nanos")),
                            ZoneId.of("UTC"));
                    val = d.format(DateTimeFormatter.ofPattern("yyyy-MM-dd'T'HH:mm:ss.SSS'Z'"));
                }
                
            } else {
                val = fixJsonTimestamps(valMap);
            }
        } else if(val instanceof List && ((List) val).size() > 0 && 
                ((List) val).get(0) instanceof Map) {
            List<Map<String, Object>> outputList = new ArrayList<>();
            for(Map item : (List<Map>)val) {
                outputList.add(fixJsonTimestamps(item));
            }
            val = outputList;
        }
        outMap.put(key, val);
    }
    return outMap;
}

Not the most ideal solution but it works for what I am doing, I think I saw someone recommend using a different timestamp class.

Hey @Benjamin McCord. Welcome to Stack Overflow and thank you for your contribution. Have you tried improving Adam's answer with an edit instead of posting your own? It would surely help to make Adam's answer complete and more people would see your contribution as Adam's has the most upvotes. — CausingUnderflowsEverywhere, Jul 28 '21 at 02:13

Sneha Mule · Answer 4 · 2022-09-22T18:37:28.527

You can convert json string to Proto using builder and json String

Example :

YourProto.Builder protoBuilder = YourProto.newBuilder();
JsonFormat.parser().merge(JsonString, protoBuilder);

If you want to ignore unknown json field then

 YourProto.Builder protoBuilder = YourProto.newBuilder();
 JsonFormat.parser()..ignoringUnknownFields().merge(JsonString, protoBuilder);

Another way is, to use mergeFrom method from ProtocolBuffer

Example :

YourProto.Builder protoBuilder = YourProto.newBuilder();
protoBuilder.mergeFrom(JsonString.getBytes());

Once it execute, you will get all the data in protoBuilder from json String

score -1 · Answer 5 · answered Feb 16 '21 at 17:32

-1

online service: https://json-to-proto.github.io/

This tool instantly converts JSON into a Protobuf. Paste a JSON structure on the left and the equivalent Protobuf will be generated to the right, which you can paste into your program. The script has to make some assumptions, so double-check the output!

answered Feb 16 '21 at 17:32

Denis Orlov

130
4

3

That creates a .proto *schema* from an example input. It sounds like the OP wants a *payload*, not a schema – Marc Gravell Feb 16 '21 at 18:34

How to convert from Json to Protobuf?

5 Answers5

Linked