Logstash CSV is not working

Question

I am trying to read CSV data in logstash, but some how logstash isn't able to split strings considering them as csv

logstash config

input {
    file {
        path => [ "/root/logstash/temp.csv" ]
        start_position => "beginning"
    }
}
filter {
    csv {
        columns => ['A','B','C','D','E']
    }
}

output {
    stdout { }
}

Test CSV file

p,q,r,s,t

p,q,r,s,t

p,q,r,s,t

p,q,r,s,t

p,q,r,s,t

p,q,r,s,t

Output of logstash

2014-04-23T13:26:53.415+0000 0.0.0.0 p,q,r,s,t

2014-04-23T13:26:53.416+0000 0.0.0.0 p,q,r,s,t

2014-04-23T13:26:53.416+0000 0.0.0.0 p,q,r,s,t

2014-04-23T13:26:53.417+0000 0.0.0.0 p,q,r,s,t

2014-04-23T13:26:53.417+0000 0.0.0.0 p,q,r,s,t

2014-04-23T13:26:53.418+0000 0.0.0.0 p,q,r,s,t

Can somebody help me with this issue?

1) I have tried replacing single quote with double quotes in Columns

2) I tried with different data

I am expecting columnar output as mentioned in this link https://blog.trifork.com/2014/01/28/using-logstash-elasticsearch-and-kibana-to-monitor-your-video-card-a-tutorial/

See this: http://stackoverflow.com/questions/31095020/use-logstash-csv-filter-doesnt-work/31159424#31159424 — Miguel Bessa, Jul 01 '15 at 11:05

score 8 · Accepted Answer · answered Apr 24 '14 at 03:00

8

In the output you need to specify the codec.

For example, with your configuraiton,

input {
   file {
       path => [ "/root/logstash/temp.csv" ]
       start_position => "beginning"
   }
}
filter {
    csv {
        columns => ['A','B','C','D','E']
    }
}

output {
    stdout { 
        codec => rubydebug
    }
}

Add the codec and then you can get what you want.

{
   "message" => [
    [0] "p,q,r,s,t"
],
  "@version" => "1",
"@timestamp" => "2014-04-24T02:57:37.099Z",
         "A" => "p",
         "B" => "q",
         "C" => "r",
         "D" => "s",
         "E" => "t"
}

answered Apr 24 '14 at 03:00

Ban-Chuan Lim

7,840
4
35
52

It worked. As per the link mentioned, I was expecting stdout to also have the columnar o/p. Thanks. – Rocky Apr 24 '14 at 04:49
@Ben, I too am just discovering logstash,and am following the same tutorial, What I noticed is that logstash wouldn't print anything on the console until the file contents changed/Updated. So is it the way it has to work or am I missing something? - Am on logstash version 1.4.2 – hussainb Aug 21 '14 at 15:41
The defualt setting of Logstash file filter will start parsing from the end. You have to specific the settings. FYI http://stackoverflow.com/questions/19546900/how-to-force-logstash-to-reparse-a-file/21372403#21372403 – Ban-Chuan Lim Aug 22 '14 at 00:47

Logstash CSV is not working

1 Answers1