We are hosting an ASP.NET Core Web API project on Amazon Elastic BeanStalk (.NET on Windows Server). Within the application, errors are logged to the file system using Serilog. This works fine in the local development machine where we can access the files through the file explorer. When deployed to the EBS, these cannot be accessed without connecting to the instance. We have configured Amazon Cloud watch to fetch the logs from the instances. However, it is fetching only the IIS logs along with some other logs specific to Elastic bean stalk. We would like to stream our custom application logs to Amazon Cloud watch. We have found the following two approaches.
Using Serilog Cloud Watch sink
We are not much interested in this as we would be tightly coupled with Amazon services. (If the other approach is not possible, we may have to rely on this)
Customizing the .ebextensions folder to include a custom config file
There are a lot of articles and SO answers which are Linux specific, but couldn't find any resource that would help for the windows instances. Following are a few links we have gone through.
https://stackoverflow.com/a/48470419/4407553
https://ajithp.com/2017/12/10/setting-up-cloudwatch-for-custom-logs-in-aws-elasticbeanstalk/
Following these articles, we came up with a .ebextensions/logs.config file that looked as shown below
files:
"C:\\Program Files\\Amazon\\ElasticBeanstalk\\config\\publogs.d\\custom_logs.conf":
content: |
[C:\CustomLogs\Error]
log_group_name = `{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "error.log"]]}`
log_stream_name = custom-logs
file = C:\CustomLogs\Error\error*
"C:\\Program Files\\Amazon\\ElasticBeanstalk\\config\\taillogs.d\\custom_logs.conf":
content: |
[C:\CustomLogs\Error]
log_group_name = `{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "error.log"]]}`
log_stream_name = custom-logs
file = C:\CustomLogs\Error\error*
We also included scripts to create the folders and to authorize the app pool to have access on them.
However, there is no change in the Cloud Watch log groups/streams. To see if the custom .conf files are created, we have connected to an instance via RDP and could see them. We could also see the custom log files being created in the specified folder. However, we couldn't find why the cloud watch agent was not picking the custom .conf file. We think that the location we were placing the file was not the correct one, but couldn't find any alternative. We have been struck here and couldn't find any way forward. Is our approach correct or are we missing anything?
Any help in this would be much appreciated. Thank you.
