20

So, I'm building a full cloud solution using kubernetes and spring boot.

My spring boot application is deployed to a container and logs directly on the console. As containers are ephemerals I'd like to send logs also to a remote logstash server, so that they can be processed and sent to elastic.

Normally I would install a filebeat on the server hosting my application, and I could, but isn't there any builtin method allowing me to avoid writing my log on a file before sending it?

Currently I'm using log4j but I see no problem in switching to another logger as long it has a "logbackappender".

Phate
  • 6,066
  • 15
  • 73
  • 138

2 Answers2

21

You can try to add logback.xml in resources folder : 

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE configuration>

<configuration scan="true">
    <include resource="org/springframework/boot/logging/logback/base.xml"/>

    <appender name="logstash" class="net.logstash.logback.appender.LogstashTcpSocketAppender">
        <param name="Encoding" value="UTF-8"/>
        <remoteHost>localhost</remoteHost>
        <port>5000</port>
        <encoder class="net.logstash.logback.encoder.LogstashEncoder">
            <customFields>{"app_name":"YourApp", "app_port": "YourPort"}</customFields>
        </encoder>
    </appender>

    <root level="INFO">
        <appender-ref ref="logstash"/>
    </root>

</configuration>

Then add logstash encoder dependency :

pom.xml

 <dependency>
        <groupId>net.logstash.logback</groupId>
        <artifactId>logstash-logback-encoder</artifactId>
        <version>4.11</version>
 </dependency>

logstash.conf

input {
    udp {
        port => "5000"
        type => syslog
        codec => json
    }
    tcp {
        port => "5000"
        type => syslog
        codec => json_lines
    }
    http {
        port => "5001"
        codec => "json"
    }
}

filter {
    if [type] == "syslog" {
        mutate {
            add_field => { "instance_name" => "%{app_name}-%{host}:%{app_port}" }
        }
    }
}

output {
    elasticsearch {
        hosts => ["${ELASTICSEARCH_HOST}:${ELASTICSEARCH_PORT}"]
        index => "logs-%{+YYYY.MM.dd}"
    }
}
Martin Choraine
  • 2,296
  • 3
  • 20
  • 37
  • 1
    Sounds fine! Could you also post the logback configuration? – Phate Aug 07 '19 at 17:24
  • 1
    I can only suggest you to take a look at `Jhipster`. I use it a lot to learn new stuff. They have down great thinks about monitoring with spring boot / elk / prometheus / grafana (https://www.jhipster.tech/monitoring/) – Martin Choraine Aug 07 '19 at 17:29
  • 1
    This version is quite outdated - https://stackoverflow.com/review/suggested-edits/32243163 - see latest version here - https://mvnrepository.com/artifact/net.logstash.logback/logstash-logback-encoder – Dave Ankin Jul 17 '22 at 04:34
5

I've just created a full working example in my repository Hope to be helpful for someone

Gavi
  • 1,300
  • 1
  • 19
  • 39