4

I am running into problems where some of our data stores are seeing a lot of throughput. We are using POJOs serialized to JSON using Jackson. What are some of the ways we can compress JSON data?

One initial thought suggested using BSON but apparently its not much smaller than JSON.

Usman Ismail
  • 17,999
  • 14
  • 83
  • 165
  • Were you able to find solution on this? I am also having same problem. I am having a JSON document but I am not sure how to serialize that JSON document using Messagepack. Any thoughts? I have already open a question on this but so far no response. If you want, I can provide a link here. – arsenal Sep 15 '13 at 07:01
  • We used the Jackson library for JSON processing and were able to use the Smile extensions to generate compressed JSON and we did not have to change any of our server side code other than some extra config. http://jackson.codehaus.org/1.8.8/javadoc/org/codehaus/jackson/smile/package-summary.html I have some benchmarks for various serialization protocols here http://techtraits.com/noproto/ – Usman Ismail Sep 15 '13 at 15:23
  • I see, so that means you didn't use messagepack at all? You just applied smile extensions on your actual JSON? – arsenal Sep 16 '13 at 03:09
  • I see... Any idea how much benefit you got after applying Smile on your JSON? Is it worth comparing with Messagepack or Avro? Or what was your reason to not go with Avro and Messagepack? – arsenal Sep 16 '13 at 03:17
  • Take a look at the links I posted they compare smile with avro and Messagepack. For the given benchmark the avro serialized to 221 bytes and smile to 341. In terms of serialization time smile was twice as fast as avro. In general my reason for going with Smile over Messagepack and avro was that it makes your project setup very messy. You have to have an upstream project to compile schema files which then have to be consumed by both client and server projects. I much prefer JSONs implicit contract. – Usman Ismail Sep 16 '13 at 04:15
  • let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/37437/discussion-between-usman-ismail-and-techgeeky) – Usman Ismail Sep 16 '13 at 04:16

2 Answers2

2

Check out CJSON.

You can see some comparisons here.

bvulaj
  • 5,023
  • 5
  • 31
  • 45
1

If you're not wedded to JSON you could try MessagePack:

MessagePack is a binary-based efficient object serialization library. It enables to exchange structured objects between many languages like JSON. But unlike JSON, it is very fast and small.

There are implementations in many languages.

jtg
  • 386
  • 1
  • 5
  • Yeah unfortunately we kind of are Wed to JSON for this project, but I have heard good things about MessagePack – Usman Ismail Mar 23 '12 at 22:11
  • After a closer look maybe I can use MessagePacks just for data storage by annotating the same objects. Do you know if MessagePack supports object hierarchies? i.e. I say ParentClass x = decode(raw data); and it should know x is actually an instance of Child class? – Usman Ismail Mar 23 '12 at 22:39
  • I'm not sure what you mean, but anything you turn into JSON you should also be able to serialize with MessagePack. More info in [this answer](http://stackoverflow.com/a/6357042/675549). If this is more about storage, could you gzip your JSON? – jtg Mar 23 '12 at 23:10