2

I'm following the steps to enable TLS/SSL encryption using td-agent and I cannot get the test to pass (https://docs.fluentd.org/v1.0/articles/in_forward#how-to-enable-tls/ssl-encryption):

1) Created the certs,

$ openssl req -new -x509 -sha256 -days 1095 -newkey rsa:2048 -keyout fluentd.key -out fluentd.crt

2) Installed them,

$ sudo mkdir -p /etc/td-agent/certs
$ sudo mv fluentd.key fluentd.crt /etc/td-agent/certs

$ sudo chown td-agent:td-agent -R /etc/td-agent/certs
$ sudo chmod 700 /etc/td-agent/certs/
$ sudo chmod 400 /etc/td-agent/certs/fluentd.key

3) Configured td-agent.conf,

$ sudo cat /etc/td-agent/td-agent.conf    

<source>
  @type forward
  <transport>
    cert_path /etc/td-agent/certs/fluentd.crt
    private_key_path /etc/td-agent/certs/fluentd.key
    private_key_passphrase testing
  </transport>
</source>

<match debug.**>
  @type stdout
</match>

4) Restarted the service,

$ sudo systemctl restart td-agent

5) When I try the test,

$ echo -e '\x93\xa9debug.tls\xceZr\xbc1\x81\xa3foo\xa3bar' | openssl s_client -connect localhost:24224

I get this on /var/log/td-agent/td-agent.log tail,

2018-05-05 12:06:08 -0300 [info]: #0 fluentd worker is now running worker=0
2018-05-05 12:06:33 -0300 [warn]: #0 incoming chunk is broken: host="127.0.0.1" msg=22                                                                        2018-05-05 12:06:33 -0300 [warn]: #0 incoming chunk is broken: host="127.0.0.1" msg=3
2018-05-05 12:06:33 -0300 [warn]: #0 incoming chunk is broken: host="127.0.0.1" msg=1                                                                         2018-05-05 12:06:33 -0300 [warn]: #0 incoming chunk is broken: host="127.0.0.1" msg=1
2018-05-05 12:06:33 -0300 [warn]: #0 incoming chunk is broken: host="127.0.0.1" msg=44
2018-05-05 12:06:33 -0300 [warn]: #0 incoming chunk is broken: host="127.0.0.1" msg=1
2018-05-05 12:06:33 -0300 [warn]: #0 incoming chunk is broken: host="127.0.0.1" msg=0
2018-05-05 12:06:33 -0300 [warn]: #0 incoming chunk is broken: host="127.0.0.1" msg=1
2018-05-05 12:06:33 -0300 [warn]: #0 incoming chunk is broken: host="127.0.0.1" msg=40                                                                        2018-05-05 12:06:33 -0300 [warn]: #0 incoming chunk is broken: host="127.0.0.1" msg=3
2018-05-05 12:06:33 -0300 [warn]: #0 incoming chunk is broken: host="127.0.0.1" msg=3                                                                         2018-05-05 12:06:33 -0300 [warn]: #0 incoming chunk is broken: host="127.0.0.1" msg="C\x91\xA4Qz\xB4\xD2\xF1\x85&2\u07F5\u0004\xC2F\x9C\xEDt\x89\u0012\xF2\u0535"
2018-05-05 12:06:33 -0300 [warn]: #0 incoming chunk is broken: host="127.0.0.1" msg=33
2018-05-05 12:06:33 -0300 [warn]: #0 incoming chunk is broken: host="127.0.0.1" msg=13
2018-05-05 12:06:33 -0300 [warn]: #0 incoming chunk is broken: host="127.0.0.1" msg=103
2018-05-05 12:06:33 -0300 [warn]: #0 incoming chunk is broken: host="127.0.0.1" msg=65
2018-05-05 12:06:33 -0300 [warn]: #0 incoming chunk is broken: host="127.0.0.1" msg=103
2018-05-05 12:06:33 -0300 [error]: #0 unexpected error on reading data host="127.0.0.1" port=59102 error_class=MessagePack::MalformedFormatError error="invalid byte"
  2018-05-05 12:06:33 -0300 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.0.2/lib/fluent/plugin/in_forward.rb:247:in `feed_each'
  2018-05-05 12:06:33 -0300 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.0.2/lib/fluent/plugin/in_forward.rb:247:in `block (2 levels) in read_messages'
  2018-05-05 12:06:33 -0300 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.0.2/lib/fluent/plugin/in_forward.rb:256:in `block in read_messages'
  2018-05-05 12:06:33 -0300 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.0.2/lib/fluent/plugin_helper/server.rb:588:in `on_read_without_connection'
  2018-05-05 12:06:33 -0300 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/cool.io-1.5.3/lib/cool.io/io.rb:123:in `on_readable'
  2018-05-05 12:06:33 -0300 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/cool.io-1.5.3/lib/cool.io/io.rb:186:in `on_readable'
  2018-05-05 12:06:33 -0300 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/cool.io-1.5.3/lib/cool.io/loop.rb:88:in `run_once'
  2018-05-05 12:06:33 -0300 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/cool.io-1.5.3/lib/cool.io/loop.rb:88:in `run'
  2018-05-05 12:06:33 -0300 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.0.2/lib/fluent/plugin_helper/event_loop.rb:84:in `block in start'
  2018-05-05 12:06:33 -0300 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.0.2/lib/fluent/plugin_helper/thread.rb:78:in `block in thread_create'

To be sure I've tested the self-signed key / crt pair with,

$ openssl rsa -modulus -noout -in fluentd.key | openssl md5
Enter pass phrase for fluentd.key:
(stdin)= b149fbd30d9192f3c3b5e445f757bbf1
$ openssl x509 -modulus -noout -in fluentd.crt | openssl md5
(stdin)= b149fbd30d9192f3c3b5e445f757bbf1

I'm running td-agent 1.0.2 on ubuntu server 16.04.

To be honest I don't know exactly where to continue..

pragmatic_programmer
  • 3,566
  • 3
  • 29
  • 36

1 Answers1

4

I was running on the same issue and after hours of investigation I was able to solve it.

The problem is on the block [transport tls] where the official documentation at https://docs.fluentd.org/v1.0/articles/in_forward is omitting the tls from the block. Adding tls on it solved the problem.

In summary, edit your in_forward to the following:

<source>
  @type forward
  <transport tls>
    cert_path ....
    private_key_path ...
    private_key_passphrase YOUR_PASSPHRASE
  </transport>

Once edited, the echo test command will succeed.

echo -e '\x93\xa9debug.tls\xceZr\xbc1\x81\xa3foo\xa3bar' |   openssl s_client -connect localhost:24224

Fluentd log output:

018-05-14 19:15:55.906208368 +0100 fluent.info: {"worker":0,"message":"fluentd worker is now running worker=0"}
2018-02-01 07:05:21.000000000 +0000 debug.tls: {"foo":"bar"}
Kelson Martins
  • 116
  • 3