How to have different log types using Serilog and ElasticSearch

Question

I am currently trying to change our system configuration to work with Serilog (instead of working with FileBeat as a shipper to LogStash)

We are also working with the log type field (which is easy to configure in the FileBeat config file) in our various queries and for indexing out logs on Elastic.

The problem is that when using Serilog we get the default type logevent, and I didn't find where I can configure it. I want to have an option to determine a specific log type per Serilog instance. At the moment I have the default type for all of my logs.

my Serilop Config is:

        var path = GetLogPath();
        var logger = new LoggerConfiguration()
            .MinimumLevel.Information()
            .Enrich.WithMachineName()
            .Enrich.WithProperty("RequestId", Guid.NewGuid())
            .WriteTo.RollingFile(
                pathFormat: path,
                outputTemplate: "{Timestamp:yyyy-MM-dd HH:mm:ss} [{Level:u4}] [{RequestId}] {Message}{NewLine}{Exception}", buffered: false, shared: true);
        logger.WriteTo.Elasticsearch(
                new ElasticsearchSinkOptions(new Uri(this.configurationService.ElasticSearchUrl())));

How can I change the log type?

EDIT

After some investigation, I found out I suppose to change the typeName filed in the LoggerConfiguration and seem that I can do so only through the AppConfig file, which again event if I will change it, the change will affect all the logger instances.

Do I miss something?

Answer 1

Updated Answer

To add Properties and values to your logger, you can use Contextual logging and Enrichment

Contextual Logger

The simplest and most direct way to attach contextual properties to a log event

First initialized your logger:

Log.Logger = new LoggerConfiguration().ReadFrom.AppSettings().CreateLogger();

Then you can create your contextual logger:

// adding Log Context
var StudentLogger = Log.Logger.ForContext<Student>();

StudentLogger.Error(/* log message */);

Or you can use correlation log entries:

// correlation Log Entries
var orderId = "some value";
var corrLog = Log.Logger.ForContext("orderId", orderId)

corrLog.Error(/* log message */);

Enrichment

In some cases we would like every event created by a logger to carry the same, fixed, property value. The Application example is one of these.

Serilog provides Enrich.WithProperty() at the level of a LoggerConfiguration for this:

Log.Logger = new LoggerConfiguration()
    .Enrich.WithProperty("Application", "e-Commerce")
    .Enrich.WithProperty("Environment", ConfigurationManager.AppSettings["Environment"])
    // Other logger configuration

---

Original Answer

There are two ways to configure Serilog:

Using API (need serilog.sinks.elasticsearch package):

var loggerConfig = new LoggerConfiguration()
    .MinimumLevel.Debug()
    .WriteTo.Elasticsearch(new ElasticsearchSinkOptions(new Uri("http://localhost:9200") ){
         AutoRegisterTemplate = true,
 });
var logger = loggerConfig.CreateLogger();

Serilog Documentation

Using configuration from AppSettings (need Serilog.Settings.AppSettings in addition to serilog.sinks.elasticsearch)

This way, you put all of your setting in AppSetting file, e.g.

<appSettings>
    <add key="serilog:using" value="Serilog.Sinks.Elasticsearch"/>
    <add key="serilog:write-to:Elasticsearch.nodeUris" value="http://localhost:9200;http://remotehost:9200"/>
    <add key="serilog:write-to:Elasticsearch.indexFormat" value="custom-index-{0:yyyy.MM}"/>
    <add key="serilog:write-to:Elasticsearch.templateName" value="myCustomTemplate"/>
  </appSettings>

And tell serilog to read the config from appSettigns

Log.Logger = new LoggerConfiguration()
  .ReadFrom.AppSettings()
  ... // Other configuration here, then
  .CreateLogger()

See: AppSetting and ElasticSearch Configure Sink

I am not sure which log event type you are referring to? In my case, I pass the object type while logging the errors:

catch (Exception ex)
{
    Logger.Error(ex, string.Format("Exception occured in Controller: {0}, Action: Post.", this.GetType()), this.GetType());

Answer 2

Just use another overload for the Elasticsearch sink:

var path = GetLogPath();
var logger = new LoggerConfiguration()
    .MinimumLevel.Information()
    .Enrich.WithMachineName()
    .Enrich.WithProperty("RequestId", Guid.NewGuid())
    .WriteTo.RollingFile(
        pathFormat: path,
        outputTemplate: "{Timestamp:yyyy-MM-dd HH:mm:ss} [{Level:u4}] [{RequestId}] {Message}{NewLine}{Exception}", buffered: false, shared: true);
logger.WriteTo.Elasticsearch(
        this.configurationService.ElasticSearchUrl(), typeName: "type");

so you don't have to specify the typeName in the appsettings and it also won't affect all instances.

Answer 3

An alternative option would be to create a global loader static class. Define multiple ILogger fields, one for error, another for diagnostic and so forth. Configure the fields inside the static constructor. Then create several public methods, WriteError(), WriteDebug(), WriteInfo() and so forth. You will be able to decide the LogEventLevel when calling the ILogger.Write(LogEventLevel.Debug, logDetail) from WriteDebug(logDetail) method for instance.