0

So I was debugging game with Fiddler, and found that the HTTP request is partially decoded and partially not decoded. The non-decoded part consists of incomprehensible unicodes:

POST https://example.com/something HTTP/1.1
Host: example.com
User-Agent: UnityPlayer/2020.3.32f1 (UnityWebRequest/1.0, libcurl/7.80.0-DEV)
Accept: */*
Accept-Encoding: deflate, gzip
Content-Type: application/octet-stream
x_acts: Duel.matching
atoken: 23128e425359819ac4253c93e72cbc944095812015c1ff83843f9f62ff1e
X-Unity-Version: 2020.3.32f1
Content-Length: 311

`   |  [  գF #   =-  a  = } E& W J &! Ġ    _  iU x_      I   cc  "`Hp!   B    x @ h~{"info":[{"n":"2","m":168,"params":{"rule":{"mode":2,"type":5}}}],"vae":"344142"}

I had the same problem in both Fiddler Classic and Fiddler Everywhere. I gave CA, and selected Decrypt option. It would be understandable if whole request is either completely decoded or completely encoded, but it is both. Is there something I'm missing? Thanks!

new
  • 329
  • 3
  • 11
  • 2
    `Accept-Encoding: deflate, gzip` ==> You need to unzip the stream, see [How to decompress gzipped HTTP response?](https://unix.stackexchange.com/questions/290546/how-to-decompress-gzipped-http-response) – Luuk May 28 '22 at 08:13
  • 1
    I think (but did not test) that you can use the "Decode" button in the fiddler toolbast to do that for you. – Luuk May 28 '22 at 08:21
  • Thank you for your response. About the first solution, I did sed -e '1,/^[[:space:]]*$/d' resp | gzip -d > resp.decompressed line with/without header, and they gave gzip: stdin: not in gzip format, gzip: stdin: unexpected end of file, respectively. Is there something else I should've done from here? – new May 28 '22 at 08:25
  • I just checked that "Decode" button in the Fiddler toolbast is toggled on. So I also expected Fiddler to do the gzip decompression for me. But for some reason, it only decompressed the last part of body, and first part of body is either not decompressed or decompressed in wrong way. – new May 28 '22 at 08:27
  • 1
    @Luuk The `Accept-Encoding` is only relevant for the response data, but not relevant for the request and the request data as shown in the question. – Robert May 28 '22 at 16:06
  • @Robert: I must have missed that, but luckily you explain that in your answer. – Luuk May 28 '22 at 16:48

1 Answers1

2

The content type of the request is set to application/octet-stream which means the request body data is binary data of any format (proprietary or standardized like PNG, ZIP, ...).

Binary data can contain plain text parts like in your example some JSON data. If you don't know the data format used for sending you can not decode the data, which means the data that Fiddler display to you are the real data that has been send. Therefore you should look at the data in hex mode and check if you can identify the format. A common binary encoding format is Google Protobuf, a decoding format that is not supported by Fiddler. If you have access to the game client which sends this data you could reverse engineer the executable and see what libraries it loads. May be one of the libraries is belongs to an encoding format. But of course the data format can be totally proprietary, then you would have to reverse engineer the executable in a tool like Ghidra, IDAPro, ... to understand what data is sent and how it is encoded.

Robert
  • 39,162
  • 17
  • 99
  • 152
  • Thank you for your answer. If the encoding format is, say, Google Protobuf, why is the last part of the body decoded? Why is only the first part of the body undecoded? – new Jun 08 '22 at 07:03
  • 1
    @Bingkongmaster Those parts are not "undecoded", they are just Strings defined by the application which sends them. Developers are often not consequent when it comes to data formats, often they just use the formats as they get them, so a JSON encoded data included as String into a binary encoding format like Protobuf is nothing unusual. I have seen even more strange combinations like JSON in XML in JSON in something else. – Robert Jun 08 '22 at 07:07
  • That is very strange to hear. Having two different encoding format in one request body. If that is true, if I decode the first part(hex values) with Google Protobuf(or other format), I will be able to get the value of first part? – new Jun 08 '22 at 07:12
  • 1
    @Bingkongmaster Usually it is a matter of efficiency. If you have library X that outputs it's data as JSON String and you want to use protobuf it doesn't make sense to decode the JSON data just to be able to encode it again as protobuf, so just use it as String and include it in your protobuf structure. – Robert Jun 08 '22 at 07:17
  • 1
    If you want to understand the data format just by looking at the data you will need luck and a lot of work. The UnityPlayer may indicate it is a game app, so you would have to understand what game actions cause what data being transmitted. Additionally you can try to decompile the app binary using an appropriate tool to get to know how the transmitted data is generated. But that will take a lot of time and skills. – Robert Jun 08 '22 at 07:17