12

I have the following configuration in my docker-compose file:

 fluentd:
    build: ./fluentd
    container_name: fluentd
    expose:
    - 24224
    - 24224/udp
    depends_on:
    - "elasticsearch"
    networks:
    -  internal

 public-site:
    build: ./public-site
    container_name: public-site
    depends_on:
    - fluentd
    logging:
      driver: fluentd
      options:
        tag: public-site
    networks:
    -  internal

networks:
  internal:

When I start the app using docker-compose up, then the webserver exists with the error message ERROR: for public-site Cannot start service public-site: failed to initialize logging driver: dial tcp 127.0.0.1:24224: connect: connection refused.

On the other hand, when I publish the ports from fluentd (ports: 24224:24224), it works. The problem is that I don't want to publish those ports on the host, since it bypasses the linux firewall (i.e. it exposes the fluentd port to everyone, see here).

This is confusing, since exposing a port should make it available for every container in the network. I am using an internal network betweem fluentd and the webserver, so I would expect that the exposed ports of fluentd are enough (which isn't the case).

When I connect to the webserver container, I can ping and resolve the fluentd container, so there is a connection. For some reasons however, at startup it won't accept a fluentd config with no published ports.

shaft
  • 2,147
  • 2
  • 22
  • 38

3 Answers3

5

The communication to 127.0.0.1 is always problematic if you're in a container. I found this explanation in the docs that performs way better than I would do:

To use the fluentd driver as the default logging driver, set the log-driver and log-opt keys to appropriate values in the daemon.json file, which is located in /etc/docker/ on Linux hosts or C:\ProgramData\docker\config\daemon.json on Windows Server. For more about +configuring Docker using daemon.json, see +daemon.json.

The following example sets the log driver to fluentd and sets the fluentd-address option.

 {
   "log-driver": "fluentd",
   "log-opts": {
     "fluentd-address": "fluentd:24224"
   }
 }

src: https://docs.docker.com/config/containers/logging/fluentd/

EDIT: this works until you want to have an application on the host communicating with the dockerized fluentd (then it's a pain)

Stefano
  • 4,730
  • 1
  • 20
  • 28
  • When I use the fluentd address, I get instead: `Cannot start service public-site: failed to initialize logging driver: dial tcp: lookup fluentd: Temporary failure in name resolution` . What can I do to help docker resolve fluentd? I have tried with `depends_on` and `links`, but both won't work. – shaft Aug 28 '19 at 14:43
  • can you do another test? try to run a `docker-compose up fluentd` wait some time and then `docker-compose up public-site`. I think there might be a problem with the resources being ready to be accessed – Stefano Aug 28 '19 at 14:47
  • Then I am getting this error: `Cannot start service public-site: failed to initialize logging driver: dial tcp: lookup fluentd on 192.168.65.1:53: no such host` – shaft Aug 28 '19 at 15:22
  • I found this issue (https://github.com/docker/compose/issues/6764) that would explain the behavior you're experiencing. – Stefano Aug 28 '19 at 15:33
2

I am facing the same error with you. After check the example config in fluent official site, I was able to connect fluentd through links.

Below is my configuration that works:

version: "3.5"

networks:
  test:

services:
  flog:
    container_name: flog
    image: mingrammer/flog:0.4.3
    command: -t stdout -f apache_common -d 1s -l
    logging:
      driver: "fluentd"
      options:
        fluentd-address: localhost:24224
    links: 
      - fluentd
    networks:
      - test

  fluentd:
    container_name: fluentd
    image: moonape1226/fluentd-with-loki-plugin:v1.13-1
    ports:
      - "24224:24224"
      - "24224:24224/udp"
    volumes:
      - ./config/fluentd/fluent.conf:/fluentd/etc/fluent.conf
    networks:
      - test
Carl Tsai
  • 380
  • 4
  • 9
2

I have facing issue, I have solve using using static ip address.

logging:
  driver: fluentd
  options: 
    fluentd-address: 172.24.0.5:24224
jacky
  • 21
  • 3