Questions tagged [msgpack]

MessagePack is a binary-based efficient object serialization library. It enables to exchange structured objects between many languages like JSON. But unlike JSON, it is very fast and small.

MessagePack is a binary-based efficient object serialization library. It enables to exchange structured objects between many languages like JSON. But unlike JSON, it is very fast and small.

Typical small integer (like flags or error code) is saved only in 1 byte, and typical short string only needs 1 byte except the length of the string itself. [1,2,3] (3 elements array) is serialized in 4 bytes using MessagePack as follows:

> require 'msgpack'
> msg = [1,2,3].to_msgpack  #=> "\x93\x01\x02\x03"
> MessagePack.unpack(msg)   #=> [1,2,3]

Scalable RPC

MessagePack-RPC is cross-language RPC library for client, server and cluster applications. Because it releases you from complicated network programming completely and provides well-designed API, you can easily implement advanced network applications with MessagePack-RPC.

require 'msgpack/rpc'
class MyHandler
  def add(x,y) return x+y end
end
svr = MessagePack::RPC::Server.new
svr.listen('0.0.0.0', 18800, MyHandler.new)
svr.run

require 'msgpack/rpc'
c = MessagePack::RPC::Client.new('127.0.0.1',18800)
result = c.call(:add, 1, 2)  #=> 3

MessagePack is available for many languages:

  • C
  • C++
  • D
  • Erlang
  • Go
  • Haskell
  • Java
  • OCaml
  • Python
  • Perl
  • PHP
  • Ruby
  • Scala
409 questions
155
votes
6 answers

Performant Entity Serialization: BSON vs MessagePack (vs JSON)

Recently I've found MessagePack, an alternative binary serialization format to Google's Protocol Buffers and JSON which also outperforms both. Also there's the BSON serialization format that is used by MongoDB for storing data. Can somebody…
Alex
  • 8,245
  • 8
  • 46
  • 55
36
votes
4 answers

MIME type for msgpack?

msgpack seems to be an extremely fast, if extremely new format for data serialisation. Does it have a recognised MIME type yet? If not, what should be used in the interim?
Tim McNamara
  • 18,019
  • 4
  • 52
  • 83
30
votes
5 answers

Distributed 1.21.8 requires msgpack, which is not installed

I'm having a problem trying to install plotly. I proceeded to upgrade anaconda using the command line on Debian 9 and I received the error message "distributed 1.21.0 requires msgpack, which is not installed". Then I tried a conda install mspack,…
Chloe Milkmaid
  • 301
  • 1
  • 3
  • 4
25
votes
1 answer

Pandas msgpack vs pickle

msgpack in Pandas is supposed to be a replacement for pickle. Per the Pandas docs on msgpack: This is a lightweight portable binary format, similar to binary JSON, that is highly space efficient, and provides good performance both on the…
Alexander
  • 105,104
  • 32
  • 201
  • 196
25
votes
6 answers

Which is the best way to compress json to store in a memory based store like redis or memcache?

Requirement : Python objects with 2-3 levels of nesting containing basic datypes like integers,strings, lists, and dicts. ( no dates etc), needs to be stored as json in redis against a key. What are the best methods available for compressing json as…
DhruvPathak
  • 42,059
  • 16
  • 116
  • 175
24
votes
1 answer

How do I read and write with msgpack?

How do I serialize / deserialize a dictionary data with msgpack?
Martin Thoma
  • 124,992
  • 159
  • 614
  • 958
23
votes
1 answer

MsgPack for compressing json data, why not using gzip/deflate?

I heard something about MsgPack that can be used for compressing json messages, but I have a question about that, why not simply using Gzip/Deflate instead of MsgPack? I think it's so better to debugging also.
Afshin Mehrabani
  • 33,262
  • 29
  • 136
  • 201
17
votes
2 answers

JavaScript to C# Numeric Precision Loss

When serializing and deserializing values between JavaScript and C# using SignalR with MessagePack I am seeing a bit of precision loss in C# on the receiving end. As an example I am sending the value 0.005 from JavaScript to C#. When the…
TGH
  • 38,769
  • 12
  • 102
  • 135
15
votes
1 answer

How to store a byte array to StackExchange.Redis?

I want to use the MessagePack, ZeroFormatter or protobuf-net to serialize/deserialize a generic list and store it in Redis using the stackexchange.redis client. Now I'm storing a JSON string with the StringSetAsync() method. But I can't find any…
tedi
  • 6,350
  • 5
  • 52
  • 67
13
votes
1 answer

How to exchange Msgpack files between Python and R?

Consider this simple example import pandas as pd mydata = pd.DataFrame({'mytime': [pd.to_datetime('2018-01-01 10:00:00.513'), pd.to_datetime('2018-01-03 10:00:00.513')], 'myvariable': [1,2], …
ℕʘʘḆḽḘ
  • 18,566
  • 34
  • 128
  • 235
13
votes
2 answers

MessagePack: fast cross-platform serializer and RPC - please share experience

Looking for some fast, simple and stable RPC library I stumbled upon MessagePack project which seems to be very good. It is also under active development. If you used it in any way, could you please share your experience? P.S. I think this question…
Vladislav Rastrusny
  • 29,378
  • 23
  • 95
  • 156
12
votes
4 answers

Using MessagePack with Android

Has someone tried to use MessagePack with an Android app? Is it possible? I have tried to use the Jar from msgpack-java and received the following Exception: Caused by: java.lang.ExceptionInInitializerError at…
arbuzz
  • 396
  • 3
  • 12
11
votes
1 answer

How to msgpack a user-defined C++ class with POD-arrays?

How it is possible to provide all three functions: msgpack_pack, msgpack_unpack and msgpack_object (also, what are meanings of them, exactly?) for a user-defined C++ class (in the same way MSGPACK_DEFINE does it for non-array POD/UD types)…
Alexander Tumin
  • 1,561
  • 4
  • 22
  • 33
11
votes
1 answer

How do I unpack and extract data properly using msgpack-c?

I'm currently trying to use msgpack in a project written in C. I'm using msgpack for the purpose of serializing the contents of a struct, which is then to be sent over the network, and deserialized back into a corresponding struct on the other…
halvdanhg
  • 198
  • 1
  • 1
  • 9
8
votes
3 answers

High performance object serialization library supporting sum types

I wonder if any of the high performance serialization libraries (like Google protocol buffers) support sum types. Sum types are tagged unions, basically the ability to say that something is either A, B, or C. Sum types are used in languages such as…
user239558
  • 6,964
  • 1
  • 28
  • 35
1
2 3
27 28