1

I'm working on some .Net framework application and i've been asked to send the logs to elasticsearch using kibana as the UI. To have something that is standardized i have to implement ECS (Elastic Common Schema).

Looking at the example we have on ECS github we only have to implement it on the following way:

enter image description here

Instead of sending to console, like we have on the example i send it to elastic search

enter image description here

The output from it, would be a nice Json object...

Maybe it is expected that on kibana we would see something like the following (Kibana - Discover):

enter image description here

Looking at that, probably the Json Object is supposed to be treated as a string and everything goes inside the message property, but that is not what i'm looking for, i want that json to be divided in many properties.

Since i'm new to Elastic stack world, i've tried to create a template inside the Index Management page and the performing there manual mappings like message._metadata.url to not treat some properties as part of the string but without success.

I'm having trouble finding useful information to solve this problem, can anyone give an hint?

UPDATE:

I found the property enableJsonLayout="true"that we can put on the target of Nlog that indeed turns whats on the Json layout as properties on ElasticSearch which is good.

enter image description here

Is this the right way to use ECS? How can i add aditional properties?

Nmaster88
  • 1,405
  • 2
  • 23
  • 65

1 Answers1

1

When you enable this enableJsonLayout="true" then it means that the configured Layout has to handle everything. For EcsLayout then you can find the documentation here:

https://github.com/elastic/ecs-dotnet/tree/master/src/Elastic.CommonSchema.NLog

EcsLayout will by default include all LogEvent Properties as metadata. See also https://github.com/NLog/NLog/wiki/How-to-use-structured-logging

But you can explicit add extra metadata-items:

<layout xsi:type="EcsLayout">
   <metadata name="MyProperty" layout="MyPropertyValue" />
</layout>
Rolf Kristensen
  • 17,785
  • 1
  • 51
  • 70
  • Cool, you're a lifesaver that metadata property data works! Where did you found that we can add that extra metadata items? I looked at some of the EcsLayout and Nlog code and documentation and did not found that. – Nmaster88 Nov 19 '20 at 18:47
  • @Nmaster88 Created pull-request to update documentation. Right now you can get a preview here: https://github.com/snakefoot/ecs-dotnet/tree/master/src/Elastic.CommonSchema.NLog – Rolf Kristensen Nov 19 '20 at 19:34
  • Thanks for the PR, i see now that probably this attribute ```[ArrayParameter(typeof(TargetPropertyWithContext), "metadata")]``` is what allows this props to be added on the xml. Btw is it possible to know what are the objective of each one? metadata i already see that is suposed to add new props inside the ```metadata``` object, but the others ```labels``` and ```tags``` i can only guess. – Nmaster88 Nov 20 '20 at 09:03
  • I noted that these ```metadata``` props, that i add through the ```web.config``` always appear even when they have no value in elasticsearch, which means that on kibana i see empty fields, is it possible to change this behaviour? not send the prop when it is not needed. – Nmaster88 Nov 20 '20 at 09:05
  • @Nmaster88 "Labels" and "Tags" are part of the ECS-schema: https://www.elastic.co/guide/en/ecs/current/ecs-base.html – Rolf Kristensen Nov 21 '20 at 11:15