1

I have been using logstash with gelf already and wanted to check out fluent input (mainly due to TCP based docker log-driver for fluent as opposed to UDP only gelf). My configuration for testing is this:

input {
  gelf {
    port => 12345
  }
  tcp {
    codec => fluent
    port => 23456
  }
}

filter {
}

output {
  stdout { codec => rubydebug { metadata => true } }
}

I can send gelf logs using:

docker run -it \
           --log-driver gelf \
           --log-opt gelf-address=udp://localhost:12345 \
           --log-opt tag=gelf-test \
        ubuntu:16.04 /bin/bash -c 'echo $(date -u +"%Y-%m-%dT%H:%M:%SZ") Hello gelf'

However the fluent version does not work:

docker run -it \
           --log-driver fluentd \
           --log-opt fluentd-address=localhost:23456 \
           --log-opt tag=fluent-test \
        ubuntu:16.04 /bin/bash -c 'echo $(date -u +"%Y-%m-%dT%H:%M:%SZ") Hello fluent'

I can verify that logstash is receiving input:

echo 'Hello TCP' | nc localhost 23456

An error occurred. Closing connection {:client=>"172.17.0.1:42012", :exception=>#, :backtrace=>["org/jruby/RubyTime.java:1073:in at'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-event-2.4.0-java/lib/logstash/timestamp.rb:32:inat'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-codec-fluent-2.0.4-java/lib/logstash/codecs/fluent.rb:41:in decode'", "org/msgpack/jruby/MessagePackLibrary.java:195:ineach'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-codec-fluent-2.0.4-java/lib/logstash/codecs/fluent.rb:40:in decode'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-tcp-3.0.6/lib/logstash/inputs/tcp.rb:153:inhandle_socket'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-tcp-3.0.6/lib/logstash/inputs/tcp.rb:143:in `server_connection_thread'"], :level=>:error}

I also disabled the fluent codec and sent fluent logs and logstash properly errors there as well and parses the fluent msgpack as message field of a regular TCP event as expected.

Received an event that has a different character encoding than you configured. {:text=>"\x94\xABfluent-test\xD2X¢鄣log\xD9\\"2017-03-10T12:58:17Z Hello fluent\r\xACcontainer_id\xD9@9cbd13eb83a02a1a4d4f83ff063d4e40b4419b7dcbcef960e4689495caa5c132\xAEcontainer_name\xAF/ecstatic_kilby\xA6source\xA6stdout\xC0", :expected_charset=>"UTF-8", :level=>:warn}

{
       "message" => "\\x94\\xABfluent-test\\xD2X¢鄣log\\xD9\\\"2017-03-10T12:58:17Z Hello fluent\\r\\xACcontainer_id\\xD9@9cbd13eb83a02a1a4d4f83ff063d4e40b4419b7dcbcef960e4689495caa5c132\\xAEcontainer_name\\xAF/ecstatic_kilby\\xA6source\\xA6stdout\\xC0",
      "@version" => "1",
    "@timestamp" => "2017-03-10T12:58:18.069Z",
          "host" => "172.17.0.1",
          "port" => 42016
}

I have no other ideas, has anybody run into this issue or have any ideas on how to debug further?

Eren Güven
  • 2,314
  • 19
  • 27

2 Answers2

0

would you please try a Fluentd instance ?, on that way would be easier to determinate where the issue is. Doing a quick view looks like Logstash Fluent codec is not working properly.

edsiper
  • 398
  • 1
  • 4
  • Fluentd as receiver works just fine so I don't think this is an issue on the docker log-driver side. I also tried using a fluentd with fluentd-output-gelf plugin as a forwarder to logstash-gelf and that works fine as well. As far as I can tell, the issue is somewhere between logstash fluentd codec and pipeline. – Eren Güven Mar 10 '17 at 16:19
  • my fluentd receives and parses events from docker just fine as well. it seems to me theres something at the logstash entry point that has gone astray. – cdaringe Nov 18 '17 at 00:00
0

Unfortunately you can't send messages from fluentd directly to logstash using the existing plugins (it's a shame really).

If you wish to do so, use this open-source plugin which sends the data directly to logstash tcp input (no need for fluentd codec) and also support sending data via secured SSL/TLS protocol.

Seen on this thread.

dorony
  • 1,008
  • 1
  • 14
  • 31