specification driven packet format parsing?

Question

this could a shot in the dark: we have a wire packet format which is in design and in flux. Is there a flexible way to specify and parse the incoming stream instead of harding coding it? Preferably language agnostic ... but not necessarily so. If there is a good reason not to do that, please enlighten me as well.

Thanks

Oliver

To avoid assuming: do you *need* a custom wire packet format? — millimoose, Dec 05 '11 at 23:50
possible duplicate of [Name me a Binary Parser. A parser for binary data](http://stackoverflow.com/questions/2214863/name-me-a-binary-parser-a-parser-for-binary-data) — Wim Coenen, Dec 05 '11 at 23:54
@Inerdial: team decision, it is too late to go back to the drawing board now. — Oliver, Dec 06 '11 at 03:29

Francis Avila · Accepted Answer · 2011-12-05T23:57:18.677

There is a wonderful Python library called Construct. This is by far the easiest way to get started quickly.

If it is more important to be language agnostic, you can limit your wire protocol to ASN.1 BER, since it's usually possible to find an encoder/decoder library for these in whatever language. For example, there's CoDec in Java and PyASN1 in Python. Warning: though powerful, ASN is laden with design-by-committee standardese and it's very hard to get started in it. This site will help.

There are also newer, lighter ASN.1-like approaches like Apache Thrift and Google Protocol Buffers.

score 1 · Answer 2 · answered Dec 05 '11 at 23:48

Google's Protocol Buffers lets you specify what you want to send, then it generates the parsing and serializing code for you.

Thrift is similar, but provides a IPC framework as well. It integrates the parsing and sockets stuff into one package.

I haven't used these libraries much, but both of them are well documented, language agnostic and are released under a permissive license – I'm sure at least one of them would suit your application.

score 1 · Answer 3 · answered Dec 06 '11 at 00:15

1

Erlang and Haskell both have excellent binary pattern matching. But for your purposes (given that you are on the JVM) why not write the connector in Scala?

answered Dec 06 '11 at 00:15

alphazero

27,094
3
30
26

specification driven packet format parsing?

3 Answers3