Can't run logstash as service

Question

I have the following logstash.config file:

input {
    file {
        path => "/home/username/log-stream-test.log"
        start_position => "beginning"
        sincedb_path => "/dev/null"
    }
}
output {
    file {
        path => "/tmp/log-stream-output.log"
    }
}

I can run logstash forcing config file (it will create output file /tmp/log-stream-output.log): /opt/bin/logstash -f /etc/logstash/conf.d/logstash.config

But when I try to run logstash as service (sudo service logstash start) it doesn't work (no output file is created). When I tail /var/log/logstash/ (with --debug option active) I get some messages like:

{:timestamp=>"2016-09-27T19:05:03.898000+0000", :message=>"Reading config file", :config_file=>"/etc/logstash/conf.d/logstash.config", :level=>:debug, :file=>"logstash/config/loader.rb", :line=>"69", :method=>"local_config"}
{:timestamp=>"2016-09-27T19:05:04.026000+0000", :message=>"Plugin not defined in namespace, checking for plugin file", :type=>"input", :name=>"file", :path=>"logstash/inputs/file", :level=>:debug, :file=>"logstash/plugin.rb", :line=>"86", :method=>"lookup"}

Am I missing something? How can I run logstash as service with this config file?

Those logs are debug logs, not error logs, so what, makes you think that it doesn't work? — Val, Sep 28 '16 at 04:43
debug log being the lowest level of logs, with message not relevant to normal use of the application. — baudsp, Sep 28 '16 at 08:16
If you have already read the log file with Logstash and if the file was not updated since, Logstash won't have anything to read, since it remembers where it stopped (cf https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html#_tracking_of_current_position_in_watched_files). — baudsp, Sep 28 '16 at 08:17
It doesn't work becase the output file isn't created. Now I'm forcing it to read from the beggining; Also, I edited the post. — Danilo, Sep 28 '16 at 12:16

score 2 · Accepted Answer · edited May 23 '17 at 10:34

2

This is almost certainly a permissions issue. When running logstash manually you run it as your user, when running it as a service, it runs as a logstash user.

Test you have access to your files:

sudo -u logstash /bin/bash

This will log you into a shell as the logstash user. Try and less the files you are trying to read.

If you have access, you will need to check your sincedb files (set the path to /dev/null) and restart, see: How to force Logstash to reparse a file?

If you don't have access, move your files somewhere where they do have access. Or grant read access to the logstash user/group.

Regards,

Artur

edited May 23 '17 at 10:34

Community

1
1

answered Sep 28 '16 at 08:59

pandaadb

6,306
2
22
41

That's right! My file /home/username/log-stream-test.log had permissions to everyone read it, but subdirectory /home/username/ has permission only to owner (username). Thanks! – Danilo Sep 28 '16 at 12:42

Can't run logstash as service

1 Answers1