Convert binary file to proto file

Question

Server returned me binary file -- https://pastebin.com/E4jxdF0W

I want to find .proto structure of the binary file

So, I added the file to the site -- https://protogen.marcgravell.com/decode/

And received

I created proto file (but don't receive valid text)

What should I change in my proto structure?

syntax = "proto2";

    message Cities {
      repeated City city = 1;
    }
    
    message City {
      optional string name = 2;
      optional string name2 = 3;
      optional string name3 = 4;
      optional string name4 = 5;
      optional string name5 = 6;
      optional string name6 = 7;
      optional int32 name7 = 8;
      optional int32 name8 = 9;
      optional int32 name9 = 10;
      optional string name10 = 11;
    }

UPDATE 1 I used

protoc --decode_raw < binary_file.protobuf

And received result

1 {
  1: "U\222\017\352c\021\021\346\204\237RT\000\020\266\010"
  2: "\320\241\320\260\320\275\320\272\321\202-\320\237\320\265\321\202\320\265\321\200\320\261\321\203\321\200\320\263"
  4 {
    1: 0x426fc1a3
    2: 0x41f286ea
  }
  5: "\320\241\320\260\320\275\320\272\321\202-\320\237\320\265\321\202\320\265\321\200\320\261\321\203\321\200\320\263"
  5: "\320\241\320\260\320\275\320\272\321\202-\320\237\320\265\321\202\320\265\321\200\320\261\321\203\321\200\320\263\320\260"
  5: "\320\241\320\260\320\275\320\272\321\202-\320\237\320\265\321\202\320\265\321\200\320\261\321\203\321\200\320\263\321\203"
  5: "\320\241\320\260\320\275\320\272\321\202-\320\237\320\265\321\202\320\265\321\200\320\261\321\203\321\200\320\263"
  5: "\320\241\320\260\320\275\320\272\321\202-\320\237\320\265\321\202\320\265\321\200\320\261\321\203\321\200\320\263\320\276\320\274"
  5: "\320\241\320\260\320\275\320\272\321\202-\320\237\320\265\321\202\320\265\321\200\320\261\321\203\321\200\320\263\320\265"
  5: "\320\241\320\260\320\275\320\272\321\202-\320\237\320\265\321\202\320\265\321\200\320\261\321\203\321\200\320\263\320\265"
  6: "sankt-peterburg"
  7: 2
}
2 {
  1: "\224\331\203\202B\303\021\346\224\031RT\000\020\266\010"
  2: "\260\336\t\360I\026I\006\217\232t\2021T\030B"
  3: "\320\237\321\217\321\202\321\221\321\200\320\276\321\207\320\272\320\260"
  4: "https://leonardo.edadeal.io/dyn/re/retailers/images/icons/sq/85cd6908f8a34aefbe66dd1948cc3d39.png"
  5: "https://leonardo.edadeal.io/dyn/re/retailers/images/logos/sq/992af68cffaa47de9ff23079648074ad.png"
  6: "eb2316"
  7: 733
  8: 55
  9: 425
  10: "5ka"
}
2 {
  1: "\224\331\205\336B\303\021\346\224\031RT\000\020\266\010"
  2: "\260\336\t\360I\026I\006\217\232t\2021T\030B"
  3: "\320\224\320\270\320\272\321\201\320\270"
  4: "https://leonardo.edadeal.io/dyn/re/retailers/images/icons/sq/146db05ebc81464cbf4872675cb2e761.png"
  5: "https://leonardo.edadeal.io/dyn/re/retailers/images/logos/sq/9c2b2c291f134929949fa4daab88252f.png"
  6: "FF8A38"
  7: 691
  8: 61
  9: 276
  10: "dixy"
}

Does this answer your question? [Decoding protobuf without schema](https://stackoverflow.com/questions/25898230/decoding-protobuf-without-schema) — grihabor, Oct 11 '20 at 11:04

score 2 · Answer 1 · answered Oct 11 '20 at 11:09

2

This is largely trial and error-based, if you can't get the schema. However, I can tell you that the root object here doesn't look like a list (repeated) - for that to be true

each sub item would have the same field number (they don't, 1 vs 2)
each sub item would have the same shape (they don't, see field 4, text vs sub-object)

So your root object has something like:

optional Foo foo = 1;
optional Bar bar = 2;

answered Oct 11 '20 at 11:09

Marc Gravell

1,026,079
266
2,566
2,900

And what are types of Foo and Bar? – swor Oct 11 '20 at 11:54
1

@swor I can't tell you the names - the binary content doesn't include that - all I know is that they are different. You'd need to reverse-engineer each separately from the payload. Presumably one of them is the type you've called `City`. The other one is a different shape, so it is something else. I can't speak Russian and I don't know what the context is, so... – Marc Gravell Oct 11 '20 at 12:00
Please, look at UPDATE 1. I found structure of protobuf and I want to create .protoc file based on this knowledge – swor Oct 11 '20 at 12:05
1

@swor which is basically the same content as the screenshot from the website. What is the actual question here? I can't tell you what the fields *are or mean*. There are some obvious text fields, and some integer fields (although I can't tell you whether they are signed or not). A couple of the fields (field 1 in the first message type, and fields 1 and 2 in the second message type) are probably `bytes`, but I can't tell you what the bytes represent. – Marc Gravell Oct 11 '20 at 12:09
1

@swor btw field 4 in the first message might be a "duration" or "timestamp" (probably a "timestamp" - does April 27th 2005 mean anything?) – Marc Gravell Oct 11 '20 at 12:12
1

@swor also, 16 bytes suggests "guid"/"uuid", but I can't tell you which endianness/layout (there are multiple conventions in use) – Marc Gravell Oct 11 '20 at 12:21
Can I eliminate the first message? Second message and others are similar? (only the firstis bad) – swor Oct 11 '20 at 12:23
2

@swor define "eliminate". None of the messages look "bad" to me. If you don't care about the data, sure: you can just ignore it. – Marc Gravell Oct 11 '20 at 12:35
"eliminate" - don't take to account "bad messages". I mean pass these messages and don't parse – swor Oct 11 '20 at 13:05
1

@swor I disagree that they're "bad", but : simply don't define any field 1 on the root message, and you don't need to worry about it (unexpected fields are either skipped or stored in an opaque form for round trip, depending on the implementation) – Marc Gravell Oct 11 '20 at 15:05

score 0 · Answer 2 · answered Apr 02 '21 at 12:12

Sorry if this is outdated, but if you look at the js running on that page there's some code for decoding those objects:

  t.id && t.hasOwnProperty("id") && e.uint32(10).bytes(t.id),
                                null != t.description && t.hasOwnProperty("description") && e.uint32(18).string(t.description),
                                null != t.imageUrl && t.hasOwnProperty("imageUrl") && e.uint32(26).string(t.imageUrl),
                                null != t.priceOld && t.hasOwnProperty("priceOld") && e.uint32(37).float(t.priceOld),
                                null != t.priceNew && t.hasOwnProperty("priceNew") && e.uint32(45).float(t.priceNew),
                                null != t.priceIsFrom && t.hasOwnProperty("priceIsFrom") && e.uint32(48).bool(t.priceIsFrom),
                                t.segmentId && t.hasOwnProperty("segmentId") && e.uint32(58).bytes(t.segmentId),
                                null != t.quantity && t.hasOwnProperty("quantity") && e.uint32(69).float(t.quantity),
                                null != t.quantityUnit && t.hasOwnProperty("quantityUnit") && e.uint32(74).string(t.quantityUnit),
                                null != t.discount && t.hasOwnProperty("discount") && e.uint32(85).float(t.discount),
                                null != t.discountUnit && t.hasOwnProperty("discountUnit") && e.uint32(90).string(t.discountUnit),
                                null != t.discountLabel && t.hasOwnProperty("discountLabel") && e.uint32(98).string(t.discountLabel),
                                null != t.discountPercent && t.hasOwnProperty("discountPercent") && e.uint32(104).uint32(t.discountPercent),
                                null != t.country && t.hasOwnProperty("country") && e.uint32(114).string(t.country),
                                null != t.dateStart && t.hasOwnProperty("dateStart") && e.uint32(122).string(t.dateStart),
                                null != t.dateEnd && t.hasOwnProperty("dateEnd") && e.uint32(130).string(t.dateEnd),
                                t.metaId && t.hasOwnProperty("metaId") && e.uint32(138).bytes(t.metaId),
                                t.compilationIds && t.compilationIds.length)
                                    for (var r = 0; r < t.compilationIds.length; ++r)
                                        e.uint32(146).bytes(t.compilationIds[r]);
                                if (t.brandIds && t.brandIds.length)
                                    for (r = 0; r < t.brandIds.length; ++r)
                                        e.uint32(154).bytes(t.brandIds[r]);

maybe that way it's easier to reverse-engineer the protobuf schema.

Convert binary file to proto file

2 Answers2