Decoding gzip response body from Packetbeat

Question

I am using Packetbeat to monitor the requests/responses into/out of Elasticsearch client nodes using the http protocol watcher on port 9200. I am sending the output of Packetbeat through Logstash, and then from there out to a different instance of Elasticsearch. We have compression support enabled in the Elasticsearch that is being monitored, so I occasionally see requests with "Accept-Encoding: gzip, deflate" headers returning responses that are gzipped. Unfortunately, I have not been able to decode any of these gzip responses using any tools I have at my disposal (including the web-based converters, the gzip command line tool, and using Zlib::GzipReader in a Logstash ruby filter script). They all report that it is not a gzip format.

Does anyone know why I can't seem to decode the gzip content?

I have provided a sample of the filter I'm using in Logstash to try to do this on the fly as the event passes through Logstash (and it always reports that http.response.body is not in gzip format).

filter {
  if [type] == "http" {
    if [http][response][headers][content-encoding] == "gzip" {
      ruby {
        init => "
          require 'zlib'
          require 'stringio'
        "
        code => "
          body = event.get('[http][response][body]').to_s
          sio = StringIO.new(body)
          gz = Zlib::GzipReader.new(sio)
          result = gz.read.to_s
          event.set('[http][response][body]', result)
        "
      }
    }
  }
}

I'm also providing a sample of the logged event here which includes the gzip content in case you would like to try to decompress it yourself:

{
  "_index": "packetbeat-6.2.3-2018.05.19",
  "_type": "doc",
  "_id": "oH0bemMB2mAXfg5euIiP",
  "_score": 1,
  "_source": {
    "server": "",
    "client_server": "",
    "bytes_in": 160,
    "bytes_out": 361,
    "@timestamp": "2018-05-19T20:33:46.470Z",
    "client_port": 55863,
    "path": "/",
    "type": "http",
    "client_proc": "",
    "query": "GET /",
    "port": 9200,
    "host": "gke-main-production-elastic-clients-5728bab3-t1z8",
    "@version": "1",
    "responsetime": 0,
    "fields": {
      "nodePool": "production-elastic-clients"
    },
    "response": "HTTP/1.1 200 OK\r\ncontent-type: application/json; charset=UTF-8\r\ncontent-encoding: gzip\r\ncontent-length: 250\r\n\r\n\u001f�\b\u0000\u0000\u0000\u0000\u0000\u0000\u0000T��n�0\u0014Fw���\u001c\u0010\u0018�����&��vH\u0016d�K������\u0010��\u000b�C\u0018����{��\u0010]\u0001�\u001aap1W\u0012�\u0018\u0017�,y)���oC�\n��A��\u001b�6/��\u001a�\u000e��\"l+�����\u001d\u000f\u0005y/���k�?�\u0005�\u0005���3���Y�_[���Mh�\u0007nzo�T����C�1�\u0011�]����\u0007H�\u0015q��)�&i��u^%iF�k�i6�ތs�c���)�9hh^�0�T2<�<���.J����x���}�:c�\u0011��=���\u001f\u0000\u0000\u0000��\u0003\u0000��.�S\u0001\u0000\u0000",
    "proc": "",
    "request": "GET / HTTP/1.1\r\nUser-Agent: vscode-restclient\r\nhost: es-http-dev.elastic-prod.svc.cluster.local:9200\r\naccept-encoding: gzip, deflate\r\nConnection: keep-alive\r\n\r\n",
    "beat": {
      "name": "gke-main-production-elastic-clients-5728bab3-t1z8",
      "version": "6.2.3",
      "hostname": "gke-main-production-elastic-clients-5728bab3-t1z8"
    },
    "status": "OK",
    "method": "GET",
    "client_ip": "10.24.20.6",
    "http": {
      "response": {
        "phrase": "OK",
        "headers": {
          "content-encoding": "gzip",
          "content-length": 250,
          "content-type": "application/json; charset=UTF-8"
        },
        "body": "\u001f�\b\u0000\u0000\u0000\u0000\u0000\u0000\u0000T��n�0\u0014Fw���\u001c\u0010\u0018�����&��vH\u0016d�K������\u0010��\u000b�C\u0018����{��\u0010]\u0001�\u001aap1W\u0012�\u0018\u0017�,y)���oC�\n��A��\u001b�6/��\u001a�\u000e��\"l+�����\u001d\u000f\u0005y/���k�?�\u0005�\u0005���3���Y�_[���Mh�\u0007nzo�T����C�1�\u0011�]����\u0007H�\u0015q��)�&i��u^%iF�k�i6�ތs�c���)�9hh^�0�T2<�<���.J����x���}�:c�\u0011��=���\u001f\u0000\u0000\u0000��\u0003\u0000��.�S\u0001\u0000\u0000",
        "code": 200
      },
      "request": {
        "params": "",
        "headers": {
          "connection": "keep-alive",
          "user-agent": "vscode-restclient",
          "content-length": 0,
          "host": "es-http-dev.elastic-prod.svc.cluster.local:9200",
          "accept-encoding": "gzip, deflate"
        }
      }
    },
    "tags": [
      "beats",
      "beats_input_raw_event"
    ],
    "ip": "10.24.41.5"
  },
  "fields": {
    "@timestamp": [
      "2018-05-19T20:33:46.470Z"
    ]
  }
}

And this is the response for that message that I receive at the client after it has been decompressed successfully by the client:

HTTP/1.1 200 OK
content-type: application/json; charset=UTF-8
content-encoding: gzip
content-length: 250

{
  "name": "es-client-7688c8d9b9-qp9l7",
  "cluster_name": "esprod",
  "cluster_uuid": "8iRwLMMSR72F76ZEONYcUg",
  "version": {
    "number": "5.6.3",
    "build_hash": "1a2f265",
    "build_date": "2017-10-06T20:33:39.012Z",
    "build_snapshot": false,
    "lucene_version": "6.6.1"
  },
  "tagline": "You Know, for Search"
}

Answer 1

I had a different situation and was able to resolve my issue. Posting it here, see if it helps your case.

I was using postman tool to test my REST API services locally. My Packetbeat used following config.

  type: http
  ports: [80, 8080, 8000, 5000, 8002]
  send_all_headers: true
  include_body_for: ["application/json", "x-www-form-urlencoded"]
  send_request: true
  send_response: true

I was getting following output in body.

I was able to get http.response.body in clear text when i added following to my postman request.

Accept-Encoding: application/json