0

We are having an issue with identifying a problem with the serilog elastic search sink library. We end up with a stack trace and error message that provides nothing meaningful. I was able to test with different credentials and everything worked. However the main issue is not having proper error messages to make troubleshooting/bugfixing more efficient.

Stack trace one, uploading to elastic search in bulk.

    2023-08-18T13:49:23.1060981Z Caught exception while preforming bulk operation to Elasticsearch: Elasticsearch.Net.ElasticsearchClientException: Request failed to execute
       at Elasticsearch.Net.Transport`1.HandleElasticsearchClientException(RequestData data, Exception clientException, IElasticsearchResponse response) in C:\Users\russc\source\git\elasticsearch-net-master\src\Elasticsearch.Net\Transport\Transport.cs:line 196
       at Elasticsearch.Net.Transport`1.FinalizeResponse[TResponse](RequestData requestData, IRequestPipeline pipeline, List`1 seenExceptions, TResponse response) in C:\Users\russc\source\git\elasticsearch-net-master\src\Elasticsearch.Net\Transport\Transport.cs:line 179
       at Elasticsearch.Net.Transport`1.Request[TResponse](HttpMethod method, String path, PostData data, IRequestParameters requestParameters) in C:\Users\russc\source\git\elasticsearch-net-master\src\Elasticsearch.Net\Transport\Transport.cs:line 61
       at Elasticsearch.Net.ElasticLowLevelClient.DoRequest[TResponse](HttpMethod method, String path, PostData data, IRequestParameters requestParameters) in C:\Users\russc\source\git\elasticsearch-net-master\src\Elasticsearch.Net\ElasticLowLevelClient.cs:line 58
       at Elasticsearch.Net.ElasticLowLevelClient.Bulk[TResponse](PostData body, BulkRequestParameters requestParameters) in C:\Users\russc\source\git\elasticsearch-net-master\src\Elasticsearch.Net\ElasticLowLevelClient.Generated.cs:line 26
       at Serilog.Sinks.Elasticsearch.ElasticsearchSink.EmitBatchChecked[T](IEnumerable`1 events)
       at Serilog.Sinks.Elasticsearch.ElasticsearchSink.EmitBatch(IEnumerable`1 events)

Stack trace two, template setup

    2023-08-18T13:49:20.9371951Z Failed to create the template. Elasticsearch.Net.ElasticsearchClientException: Request failed to execute
       at Elasticsearch.Net.Transport`1.HandleElasticsearchClientException(RequestData data, Exception clientException, IElasticsearchResponse response) in C:\Users\russc\source\git\elasticsearch-net-master\src\Elasticsearch.Net\Transport\Transport.cs:line 196
       at Elasticsearch.Net.Transport`1.FinalizeResponse[TResponse](RequestData requestData, IRequestPipeline pipeline, List`1 seenExceptions, TResponse response) in C:\Users\russc\source\git\elasticsearch-net-master\src\Elasticsearch.Net\Transport\Transport.cs:line 179
       at Elasticsearch.Net.Transport`1.Request[TResponse](HttpMethod method, String path, PostData data, IRequestParameters requestParameters) in C:\Users\russc\source\git\elasticsearch-net-master\src\Elasticsearch.Net\Transport\Transport.cs:line 61
       at Elasticsearch.Net.ElasticLowLevelClient.DoRequest[TResponse](HttpMethod method, String path, PostData data, IRequestParameters requestParameters) in C:\Users\russc\source\git\elasticsearch-net-master\src\Elasticsearch.Net\ElasticLowLevelClient.cs:line 58
       at Elasticsearch.Net.ElasticLowLevelClient.IndicesExistsTemplateForAll[TResponse](String name, IndexTemplateExistsRequestParameters requestParameters) in C:\Users\russc\source\git\elasticsearch-net-master\src\Elasticsearch.Net\ElasticLowLevelClient.Generated.cs:line 1202
       at Serilog.Sinks.Elasticsearch.ElasticsearchSinkState.RegisterTemplateIfNeeded()

Code to capture/output errors by the sink explicitly.

    Serilog.Debugging.SelfLog.Enable(Console.Error);
            
    ElasticsearchSinkOptions elasticSearchSinkOptions = new(new Uri(serviceSettings.Aws.ElasticSearch.Url))
    {
        IndexFormat = serviceSettings.Aws.ElasticSearch.Index,
        FailureCallback = e =>
        {
            if (e.Exception != null)
            {
                Console.Error.WriteLine($"{e.RenderMessage()}");
                Console.Error.WriteLine(e.Exception);
            }
            else
            {
                Console.WriteLine(e.RenderMessage());
            }
        },
        EmitEventFailure = EmitEventFailureHandling.RaiseCallback | EmitEventFailureHandling.WriteToSelfLog,
        TypeName = "_doc",
        AutoRegisterTemplate = true,
        InlineFields = false,
        MinimumLogEventLevel = defaultLogEventLevel
    };
grim
  • 1
  • 2

0 Answers0