I am following this link (Link) to display live weather data on Kibana. I am using logstash to stash the live data into elasticsearch.
I used the logstash configuration explained in the pdf.
Here is my logstash configuration:
input {
exec {
command => "curl --silent -L ’http://api.openweathermap.org/data/2.5/weather?id=5490223&appid=my_api_key&units=metric’"
codec => "json"
interval => 600
}
}
filter {
mutate {
remove_field => ["@version" ,"command" ,"host" ,"cod" ,"id" ,"base" ,"coord" ,"sys" ,"dt"]
}
split { field => "weather" }
}
output {
elasticsearch {
hosts => ["localhost:9200"]
index => "weather-%{+YYYY.MM.dd}"
}
}
When I execute, I get an error like this
PS E:\elk\logstash\bin> .\logstash -f weather_data.conf --config.test_and_exit
Sending Logstash's logs to E:/elk/logstash/logs which is now configured via log4j2.properties
[2018-07-17T15:59:55,097][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
Configuration OK
[2018-07-17T15:59:57,966][INFO ][logstash.runner ] Using config.test_and_exit mode. Config Validation Result: OK. Exiting Logstash
PS E:\elk\logstash\bin> .\logstash -f weather_data.conf
Sending Logstash's logs to E:/elk/logstash/logs which is now configured via log4j2.properties
[2018-07-17T16:00:43,198][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2018-07-17T16:00:44,089][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"6.3.1"}
[2018-07-17T16:00:49,292][INFO ][logstash.pipeline ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>8, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50}
[2018-07-17T16:00:50,101][INFO ][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://localhost:9200/]}}
[2018-07-17T16:00:50,119][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://localhost:9200/, :path=>"/"}
[2018-07-17T16:00:50,488][WARN ][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>"http://localhost:9200/"}
[2018-07-17T16:00:50,596][INFO ][logstash.outputs.elasticsearch] ES Output version determined {:es_version=>6}
[2018-07-17T16:00:50,603][WARN ][logstash.outputs.elasticsearch] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>6}
[2018-07-17T16:00:50,650][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//localhost:9200"]}
[2018-07-17T16:00:50,687][INFO ][logstash.outputs.elasticsearch] Using mapping template from {:path=>nil}
[2018-07-17T16:00:50,819][INFO ][logstash.outputs.elasticsearch] Attempting to install template {:manage_template=>{"template"=>"logstash-*", "version"=>60001, "settings"=>{"index.refresh_interval"=>"5s"}, "mappings"=>{"_default_"=>{"dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword", "ignore_above"=>256}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date"}, "@version"=>{"type"=>"keyword"}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}}
[2018-07-17T16:00:50,921][INFO ][logstash.inputs.exec ] Registering Exec Input {:type=>nil, :command=>"curl --silent -L 'http://api.openweathermap.org/data/2.5/weather?id=5490223&appid=6fe2c29d12b0cced47f3efa361dc1fe8&units=metric'", :interval=>600, :schedule=>nil}
[2018-07-17T16:00:50,989][INFO ][logstash.pipeline ] Pipeline started successfully {:pipeline_id=>"main", :thread=>"#<Thread:0x59b6b4e7 run>"}
[2018-07-17T16:00:51,154][INFO ][logstash.agent ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
'appid' is not recognized as an internal or external command,
operable program or batch file.
'units' is not recognized as an internal or external command,
operable program or batch file.
[2018-07-17T16:00:51,890][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
'appid' is not recognized as an internal or external command,
operable program or batch file.
'units' is not recognized as an internal or external command,
operable program or batch file.
What wrong am I doing? Can someone please help me?
