0

I have the below log file which I wanted to index into elasticsearch.

The log file content are :

64.242.88.10 - - [08/Mar/2004:08:29:36 -0800] "GET /twiki/bin/view/TWiki/TWikiCategoryTable HTTP/1.1" 200 3729
219.95.17.51 - - [08/Mar/2004:08:29:57 -0800] "GET / HTTP/1.1" 200 3169
212.92.37.62 - - [08/Mar/2004:08:30:25 -0800] "GET /twiki/bin/view/Main/SpamAssassinDeleting HTTP/1.1" 200 5543
212.92.37.62 - - [08/Mar/2004:08:31:37 -0800] "GET /twiki/bin/view/Main/DCCAndPostFix HTTP/1.1" 200 5253
10.0.0.153 - - [08/Mar/2004:08:32:24 -0800] "GET / HTTP/1.1" 304 -
10.0.0.153 - - [08/Mar/2004:08:32:27 -0800] "GET /twiki/bin/view/Main/WebHome HTTP/1.1" 200 10419

The logstash configuration file is :

input {
  file {
    path => "C:/Users/abhijitb/Downloads/apache-access_log/access_log/access_log.log"
    start_position => "beginning"
    sincedb_path => "logger.txt"
  }
}

filter {
 grok {
   match=> {
    "message" => '%{IPORHOST:clientip} - - \[%{HTTPDATE:timestamp}\] "%{WORD:verb} %{DATA:request} HTTP/%{NUMBER:httpversion}" %{NUMBER:response:int} %{NUMBER:bytes}'
    }
  }

  date {
    match => [ "timestamp", "dd/MMM/YYYY:HH:mm:ss Z" ]
  }

  geoip {
    source => "clientip"
  }
}

output {
   stdout { codec => rubydebug { metadata => true } }
   elasticsearch {
     hosts => "http://localhost:9200"
     index => "logstash-logs-1"
  }
}

When I run the command from the logstash as

logstash -f logstashLogFile.conf

nothing happens..

Logstash Running

I tested the same using Test grok patterns, it shows all the result.. Grok Test

Grok Result

Abhijit Bashetti
  • 8,518
  • 7
  • 35
  • 47
  • 1
    If logstash has already read a file, it will remeber what has been read and will only pick up new lines added to the file. Delete the sincedb file. See also https://stackoverflow.com/questions/19546900/how-to-force-logstash-to-reparse-a-file – baudsp Feb 07 '19 at 11:13
  • 1
    removed the sincedb files from the location "C:\logstash-6.5.4\logstash-6.5.4\data\plugins\inputs\file"...but no success...also removed the logger.txt... – Abhijit Bashetti Feb 07 '19 at 11:35
  • 1
    I don't have any other idea. – baudsp Feb 07 '19 at 12:27
  • 1
    Appreciate your help...I will give a try again... – Abhijit Bashetti Feb 07 '19 at 12:31

0 Answers0