0

We're trying to configure filtered logging within our Azure App, we have configured our app to save to blob storage, and we can access and view these fine.

The issue we're having is currently azure logs all information level logs across all services, so we're getting information level logs for routing, entity framework, etc, where for day to day logging all we really need is what our manual logging in the controllers are doing.

I was always under the impression we could filter logs in our appsettings.json like so:

"Logging": {
"LogLevel": {
  "Default": "Information",
  "Microsoft": "None"
}

But this get's ignore in Azure and all levels of logs continue for everything, tried the same settings in appsettings.development.json and had the same outcome.

I also read in the docs we can add filters to Azure Application Insights logs programmatically, but this also doesn't seem to be working (have also set the application insights connection string in env variable)>

builder.Services.AddApplicationInsightsTelemetry();

builder.Host.ConfigureLogging(log =>
{
    log.ClearProviders();
    log.AddFilter<ApplicationInsightsLoggerProvider>("MyAppName", LogLevel.Information);
    log.AddFilter<ApplicationInsightsLoggerProvider>("Microsoft", LogLevel.Error);
    log.AddAzureWebAppDiagnostics();
});

This also did not provide the solution we had hoped. Is there a specific way this needs to be configured? Thanks in advance!

JimDev
  • 39
  • 1
  • 1
  • 5
  • Check the "App Service Logs" configuration in the App Service. As a guess, I think it might be set to "Information" there. – Kirk Larkin Aug 08 '22 at 15:38
  • You do know that you don't need to use AppInsights, right? Have you considered other MEL backends, like Serilog? – Dai Aug 08 '22 at 15:38
  • @KirkLarkin AFAIK the "Application logging" setting in Azure Portal controls the `System.Diagnostics.Trace` listeners - it only affects MEL and other logging systems if they choose to integrate with the .NET Trace system. – Dai Aug 08 '22 at 15:42
  • @Dai: It affects MEL when you use `AddAzureWebAppDiagnostics` (I'm assuming MEL = Microsoft.Extensions.Logging). – Kirk Larkin Aug 08 '22 at 15:49
  • Thanks @KirkLarkin already set there to Information, but it logs all information level events from all providers, only want to show application level events, no options to filter there beyond the level. – JimDev Aug 08 '22 at 15:51
  • @JimDev: I don't think you can override that with the `AddAzureWebAppDiagnostics` stuff. The docs suggests otherwise, but I dug into the code for this recently and the provider just doesn't appear to read anything other than that global level setting. – Kirk Larkin Aug 08 '22 at 15:52
  • @Dai Good to know, thank you, would prefer to stick with the default MS loggers if possible, just moved away from NLog and Log4Net and don't want to keep switching packages. Do you know if what I am looking for is possible in Azure? – JimDev Aug 08 '22 at 15:53
  • @KirkLarkin Are you suggesting that what I am looking for isn't possible, and azure will always provide all global level information logs, regardless of what we programmatically include in appsettings/program.cs? – JimDev Aug 08 '22 at 15:55
  • I use MEL with Serilog as a back-end, logging to the local filesystem. You only need to reference Serilog in the entrypoint project for setup, otherwise _all_ logging goes through `Microsoft.Extensions.Logging.ILogger`. For example: https://stackoverflow.com/questions/71599246/how-to-configure-and-use-serilog-in-asp-net-core-6 - I have my own custom in-app code that reads-back the saved JSON-per-line files - as well as reading inter-site log-files via Kudu. It's worked great for me for the past 3+ years. YMMV of course. – Dai Aug 08 '22 at 15:55
  • @Dai apologies if I have the wrong end of your advice, but does that only provide local logging to the file system, and still doesn't allowing filtering at the azure portal/blob storage level? – JimDev Aug 08 '22 at 15:58
  • @JimDev I don't know - so I asked Microsoft just now: https://github.com/MicrosoftDocs/azure-docs/issues/96830 – Dai Aug 08 '22 at 15:59
  • @Dai thanks for your help but this is sorted now. – JimDev Aug 10 '22 at 13:08

1 Answers1

1

Solved this now, thanks for everyone that added comments. For anyone else coming across this, you have to ensure your appsettings.json includes the full range of Logging profiles. You would think the default LogLevel would work across all applications but this is not the case.

 "Logging": {
"LogLevel": { // No provider, LogLevel applies to all the enabled providers.
  "DigitalAppraisalSystem": "Information",
  "Microsoft": "Warning",
  "Microsoft.Hosting.Lifetime": "Warning"
},
"Debug": { // Debug provider.
  "LogLevel": {
    "DigitalAppraisalSystem": "Information",
    "Microsoft": "Warning",
    "Microsoft.Hosting.Lifetime": "Warning"
  }
},
"AzureAppServicesFile": {
  
  "LogLevel": {
    "DigitalAppraisalSystem": "Information",
    "Microsoft": "Warning",
    "Microsoft.Hosting.Lifetime": "Warning"
  }
},
"AzureAppServicesBlob": {
  
  "LogLevel": {
    "DigitalAppraisalSystem": "Information",
    "Microsoft": "Warning",
    "Microsoft.Hosting.Lifetime": "Warning"
  }
},
"ApplicationInsights": {
  "LogLevel": {
    "DigitalAppraisalSystem": "Information",
    "Microsoft": "Warning",
    "Microsoft.Hosting.Lifetime": "Warning"
  }
}

Alongside this, Program.cs also needs configuring of those logs, specifically:

builder.Host.ConfigureLogging((hostingContext, log) =>
{
    log.ClearProviders();
    log.AddAzureWebAppDiagnostics();
    log.AddConfiguration(hostingContext.Configuration.GetSection("Logging"));
    log.AddConsole();
    log.AddDebug();
    log.AddEventSourceLogger();
});

Now log filtering in Azure is working as intended, and blob storage isn't filled with routing traces and everything else that was causing issues with having visibility of our own user's actions.

JimDev
  • 39
  • 1
  • 1
  • 5