6

Is it possible to dump(view) data written in PB format without any knowledge about types used to write that data?

I've found https://stackoverflow.com/a/10253515/883738 that

Briefly, on the wire, protobufs are encoded as 3-tuples of , where the key is the field number assigned to the field in the .proto schema. The type is one of . It contains just enough information to decode the value of the 3-tuple, namely it tells you how long the value is.

What is my final goal is to write extension for Fiddler2 to see what's being sent/received in PB format.

Community
  • 1
  • 1
Alex Sorokoletov
  • 3,102
  • 2
  • 30
  • 52

2 Answers2

6

There is a wireshark tool for this, iirc.

The problem here is that the protobuf format is ambiguous if you don't know the schema:

  • a fixed-32 could be a float or a signed or unsigned integer (32 bit)
  • a fixed-64 could be a double or a signed or unsigned integer (64 bit)
  • a varint could be a signed or unsigned integer, a zig-zag integer, or a boolean
  • a string could be a utf-8 string, a packed array of primitives, a sub-message, or raw bytes

In fact, the only unambiguous tokens are start/end group, and they are semi-deprecated!

So: it is sort of doable, but you might need to present multiple interpretations of the same data

You can also only list field numbers: there are no member-names in the binary format

Marc Gravell
  • 1,026,079
  • 266
  • 2,566
  • 2,900
  • Wireshark tool is awesome, the only issue it doesn't play well with HTTPS. Multiple representations is great idea. So, is it possible using your library? – Alex Sorokoletov Sep 08 '12 at 16:16
  • @Alex oh, sure you could use ProtoReader easily enough - just basically loop while reader.ReadFieldHeader()>0 but that won't (and can't) tell you whether something is a float vs a signed int vs an unsigned int, etc. Utf-8 you can probably do by trying it and seeing if it works, but ... – Marc Gravell Sep 08 '12 at 16:24
  • Thanks! One more question. Can PB data start with 0? – Alex Sorokoletov Sep 08 '12 at 16:35
  • 1
    @Alex no it cannot; the first token is a field-number (left-shifted and combined with the wire-type); field-numbers are strictly positive: so no, a zero start is not legal – Marc Gravell Sep 08 '12 at 17:01
  • is it possible to discuss this with you directly? Found that sometimes PB-apps send 0 as leading byte – Alex Sorokoletov Sep 19 '12 at 18:09
  • 1
    @alex my email is on my profile page. A leading zero byte is not technically legal in protobuf, but I've heard of people using it as a message separator when sending multiple messages on a single stream – Marc Gravell Sep 19 '12 at 20:44
2

There is a Fiddler PB implementation here (haven't used it): https://github.com/SecurityInnovation/ProtoMiddler

Mark
  • 2,926
  • 3
  • 28
  • 31