How to stop logstash to write logstash logs to syslog?

Question

I have my logstash configuration in my ubuntu server which reads data from the postgres database and send the data to elastic search. I have configured a schedule at each 15 minutes the logstash will look the postgres table, if there is any change in the table it sends the data to elastic search.

But each time the logstash is also sending the logs to syslog which I does not need. Because of logstash my syslog file consumes more memory.

So how to stop logstash to send its logs to syslog. Is there is any configuration in logstash.yml to avoid sending logs to syslog.

I referred many sites in online in which they said to remove below line from the configuration.

stdout { codec => rubydebug }

But I don't have this line.

In my output I just send my data to elastic search which I brought from AWS.

Is there is a way to stop logstash to sending its logs to syslog?

See if syslog plugin is installed and remove it using: `/bin/logstash-plugin remove logstash-output-syslog` — ABusy_developerT, Oct 23 '19 at 09:49
I don't have any directory "/bin/logstash-plugin". Is there is any other way? — Prakash, Oct 23 '19 at 10:03
Do you have a .conf file where configuration for the pipeline is saved? — ABusy_developerT, Oct 23 '19 at 10:42
I created a separate folder in /etc/logstash. Inside my folder I have a two conf file which holds my logic. — Prakash, Oct 23 '19 at 10:47
would you be able to share the configurations in the conf files.. — ABusy_developerT, Oct 23 '19 at 11:00

score 9 · Answer 1 · answered Dec 12 '19 at 07:24

disable the rootLogger.appendRef.console in log4j

The logfiles that logstash itself produces are created through log4j, one stream goes by default to the console. Syslog will write to consolelogs to the syslog file itself. In the Ubuntu version of logstash this is configured in the file name/etc/logstash/log4j2.properties

In the default configuration there is a line that starts with

rootLogger.appenderRef.console

If you add a # in front of the line and restart logstash. The logfiles that logstash creates will stop going to syslog

service logstash restart

The other rootLogger that uses the RollingFileAppender should still write logmessages from logstash itself (so not the messages that are being processed by your pipeline) to

/var/log/logstash/logstash-plain.log

It's easy to confuse the logfiles that logstash creates with the messages that you process, especially if they get mixed by the logstash-output-stdout or logstash-output-syslog plugins. This is not applicable to you because you use the logstash-output-elasticsearch plugin that writes to elasticsearch.

The log4j.properties file gets skipped if you run logstash from the commandline, in Ubuntu. It's a nice way of testing your pipeline in a terminal, you can run multiple logstash instances in parallel (e.g. the service and a commandline test pipeline)

/usr/share/logstash/bin/logstash -f your_pipeline.conf

mik3fly-4steri5k · Accepted Answer · 2021-04-27T12:03:44.180

5

To avoid write to syslog, check your pipelines and log4j.properties files.

In your pipelines files, remove all occurences of this :

 stdout { codec => rubydebug }

And in your log4j.properties files comment this line :

rootLogger.appenderRef.console

edited Apr 27 '21 at 12:03

answered Feb 27 '20 at 12:38

mik3fly-4steri5k

712
3
15
32

score 0 · Answer 3 · answered Jun 23 '23 at 08:54

Not sure why it's happened but we solved by disabling the rootLogger.appenderRef.console.ref = ${sys:ls.log.format}_console.

Steps:

vim /etc/logstash/log4j2.properties
update rootLogger.appenderRef.console.ref = ${sys:ls.log.format}_console to #rootLogger.appenderRef.console.ref = ${sys:ls.log.format}_console

log4j.properties should look like the following:

rootLogger.level = ${sys:ls.log.level}
#rootLogger.appenderRef.console.ref = ${sys:ls.log.format}_console
rootLogger.appenderRef.rolling.ref = ${sys:ls.log.format}_rolling
rootLogger.appenderRef.routing.ref = pipeline_routing_appender

https://github.com/elastic/logstash/blob/main/config/log4j2.properties

Please make sure that your disable ALL logstash instance.

ps -ef |grep logstash

How to stop logstash to write logstash logs to syslog?

3 Answers3