Could not communicate to Elasticsearch

Question

I am trying to send my node app logs to fluentd to elasticsearch to kibana, but having a problem connecting fluentd with elasticsearch with docker. I want to dockerize this efk stack. I have attached the folder structure and shared relevant files. Following is the folder structure:

Error

Could not communicate to Elasticsearch, resetting connection and trying again. Connection refused - connect(2) for 172.20.0.2:9200 (Errno::ECONNREFUSED)

fluent.conf:

  @type forward
  port 24224
  bind 0.0.0.0
</source>
<match *.**>
  @type copy
  <store>
    @type elasticsearch
    host elasticsearch
    port 9200
    user elastic
    password pass
  </store>
</match>

DockerFile

FROM fluent/fluentd:v1.15-1
USER root
RUN gem install elasticsearch -v 7.6.0
# RUN gem install fluent-plugin-elasticsearch -v 7.6.0
RUN gem install fluent-plugin-elasticsearch -v 4.1.1
RUN gem install fluent-plugin-rewrite-tag-filter
RUN gem install fluent-plugin-multi-format-parser
USER fluent

Docker-compose.yml

version: '3'
services:
  fluentd:
    build: ./fluentd
    container_name: loggingFluent
    volumes:
      - ./fluentd/conf:/fluentd/etc
      # - ./fluentd/conf/fluent.conf:/fluentd/etc/fluent.conf
    ports:
      - "24224:24224"
      - "24224:24224/udp"
    links:
      - elasticsearch
    depends_on:
      - elasticsearch
      - kibana  
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.8.1
    container_name: elasticsearch-Logging
    ports:
      - 9200:9200
    expose:
      - 9200  
    environment:
      discovery.type: 'single-node'
      ES_JAVA_OPTS: '-Xms1024m -Xmx1024m'
      xpack.security.enabled: 'true'
      ELASTIC_PASSWORD: 'pass'
  kibana:
    image: docker.elastic.co/kibana/kibana:7.8.1
    container_name: kibana-Logging
    volumes:
      - ./kibana.yml:/usr/share/kibana/config/kibana.yml
    ports:
      - 5601:5601
    depends_on:
      - elasticsearch     
    links:
      - elasticsearch

Maybe I am missing some with docker networking because I am using docker for the first time I have checked the ports exposed by docker containers and they are fine. I have done this without docker and have used the same settings but having a problem doing it with docker. Looking forward to seeing your responses. Thank you very much.

What exactly is the problem? Do you have any kind of error msg? — Calipee, Nov 02 '22 at 10:01

score 0 · Accepted Answer · answered Nov 03 '22 at 06:55

Adding a username to the elastic search environment solved the issue :

elasticsearch-environment

environment:
      discovery.type: 'single-node'
      ES_JAVA_OPTS: '-Xms1024m -Xmx1024m'
      xpack.security.enabled: 'true'
      ELASTIC_PASSWORD: 'pass'
      ELASTIC_USERNAME: 'elastic'

Could not communicate to Elasticsearch

1 Answers1