Thrift - converting from simple JSON

Question

I created the following Thrift Object:

struct Student{
        1: string id;
        2: string firstName;
        3: string lastName
}

Now I would like to read this object from JSON. According to this post this is possible

So I wrote the following code:

String json = "{\"id\":\"aaa\",\"firstName\":\"Danny\",\"lastName\":\"Lesnik\"}";
    StudentThriftObject s = new StudentThriftObject();
    byte[] jsonAsByte = json.getBytes("UTF-8");
    TMemoryBuffer memBuffer = new TMemoryBuffer(jsonAsByte.length);
    memBuffer.write(jsonAsByte);

    TProtocol proto = new TJSONProtocol(memBuffer);
    s.read(proto);

What I'm getting is the following exception:

Exception in thread "main" org.apache.thrift.protocol.TProtocolException: Unexpected character:i
    at org.apache.thrift.protocol.TJSONProtocol.readJSONSyntaxChar(TJSONProtocol.java:322)
    at org.apache.thrift.protocol.TJSONProtocol.readJSONInteger(TJSONProtocol.java:698)
    at org.apache.thrift.protocol.TJSONProtocol.readFieldBegin(TJSONProtocol.java:837)
    at com.vanilla.thrift.example.entities.StudentThriftObject$StudentThriftObjectStandardScheme.read(StudentThriftObject.java:486)
    at com.vanilla.thrift.example.entities.StudentThriftObject$StudentThriftObjectStandardScheme.read(StudentThriftObject.java:479)
    at com.vanilla.thrift.example.entities.StudentThriftObject.read(StudentThriftObject.java:413)
    at com.vanilla.thrift.controller.Main.main(Main.java:24)

Am I missing something?

First just parse your JSON into Maps and Lists, then dump that and understand it. Then figure out how to extract the data you want. — Hot Licks, Nov 28 '13 at 18:55
I'm sorry, but I have not goy your point. What json should I parse. I have very simple thrift structure there is no needs for Maps and Lists. — danny.lesnik, Nov 28 '13 at 19:18
The problem is that you're using tools that are more complicated than you need (and understand). Start simple. — Hot Licks, Nov 28 '13 at 22:11

JensG · Accepted Answer · 2013-11-29T17:47:33.423

You are missing the fact, that Thrift's JSON is different from yours. The field names are not written, instead the assigned field ID numbers are written (and expected). Here's an example for Thrift's JSON protocol:

[1,"MyService",2,1,{"1":{"rec":{"1":{"str":"Error: Process() failed"}}}}]

In other words, Thrift is not intended to parse any kind of JSON. It supports a very specific JSON format as one of the possible transports.

However, depending on what the origin of your JSON data is, Thrift can possibly still help you out, if you are able to use it on both sides. In that case, write an IDL to describe the data structures, feed it to the Thrift compiler and integrate both the generated code and the neccessary parts of the library with your projects.

If the origin of the JSON lies outside of your reach, or if the JSON format cannot be changed for some reason, you need to find another way.

Format and semantics are different beasts

To some extent, the whole issue can be compared with XML: There is one general XML syntax, which tells us how we have to fomat things so any standard conformant XML processor can read them.

But knowing the rules of XML is only half the answer, if we get a certain XML file from someone. Even if our XML parser can read the file successfully, because it is well-formed XML, we need to know the semantics of the data to really make use of what's within that file: Is it a customer data record? Or is it a SOAP envelope? Maybe a configuration file?

That is where DTDs or XML Schema come into play, they exist to describe the contents of the XML data. Without knowing the logical structure you are lost, because there are myriads of possible ways to express things in XML. And exactly the same is true with JSON, except that JSON schema descriptions are less commonly used.

"So you mean, we need just a way to tell Thrift how the JSON is organized?"

No, because the purpose and idea behind Thrift is to have a framework to de/serialize things and/or implement RPC servers and clients as efficiently as possible. It is not intended to have a general purpose file parser. Instead, Thrift reads and speaks only its own set of formats, which are plugged into the architecture as protocols: Thrift Binary, Thrift JSON, Thrift Compact, and a few more.

What you could do: In addition to what I said at in the first section of my answer, you may consider writing your own custom Thrift protocol implementation to support your particular JSON format of choice. It is not that hard, and worth a try.

It looks like twitter are serializing json using the same approach: https://github.com/twitter/commons/blob/master/src/java/com/twitter/common/io/ThriftCodec.java the only difference is that they are using thrift 0.5.0 and I'm using 0.9.0. I'll try to generate binaries and run my code on this version. I've a feeling that behavior has been changed since 0.5 release. — danny.lesnik, Nov 28 '13 at 22:31
@danny.lesnik: Have you read the comment right on top? It says "*A Codec that can encode and decode thrift structs*" - it does not say "*A Codec that can encode and decode any JSON struct*". — JensG, Nov 29 '13 at 17:23

Thrift - converting from simple JSON

1 Answers1

Linked