AWS Elastic Beanstalk: Custom Cloudwatch Logs not showing despite IAM permissions and custom config

Question

I've been having issues setting up custom log file streaming to CloudWatch from my AWS Elastic Beanstalk project. For reference, I've tried the suggestions in "AWS Elastic Beanstalk: Add custom logs to CloudWatch?" to no avail. Essentially, when I download the logs off the host, the log files I want are located at /var/log/containers/api-1c080332ba3f-stdouterr.log, /var/log/containers/nginx-a5057f87f4cf-stdouterr.log, and /var/log/containers/web-0a2e0762e8f0-stdouterr.log (where the numbers change on each update).

I added a new custom log config file in .ebextensions/log.config (following this for reference):

packages:
  yum:
    awslogs: []

files:
  "/etc/awslogs/awscli.conf" :
    mode: "000600"
    owner: root
    group: root
    content: |
      [plugins]
      cwlogs = cwlogs
      [default]
      region = `{"Ref":"AWS::Region"}`

  "/etc/awslogs/awslogs.conf" :
    mode: "000600"
    owner: root
    group: root
    content: |
      [general]
      state_file = /var/lib/awslogs/agent-state

  "/etc/awslogs/config/logs.conf" :
    mode: "000600"
    owner: root
    group: root
    content: |
      [/var/log/containers/nginx-stdouterr.log]
      log_group_name = `{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "var/log/containers/nginx-stdouterr.log"]]}`
      log_stream_name = {instance_id}
      file = /var/log/containers/nginx*

      [/var/log/containers/web-stdouterr.log]
      log_group_name = `{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "var/log/containers/web-stdouterr.log"]]}`
      log_stream_name = {instance_id}
      file = /var/log/containers/web*

      [/var/log/containers/api-stdouterr.log]
      log_group_name = `{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "var/log/containers/api-stdouterr.log"]]}`
      log_stream_name = {instance_id}
      file = /var/log/containers/api*

commands:
  "01":
    command: systemctl enable awslogsd.service
  "02":
    command: systemctl restart awslogsd

I've additionally added this policy to the service and ec2 roles:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "logs:CreateLogStream",
                "logs:CreateLogGroup",
                "logs:PutLogEvents",
                "logs:DescribeLogGroups",
                "logs:DescribeLogStreams"
            ],
            "Resource": [
                "*"
            ]
        }
    ]
}

Finally, I don't see any errors in /var/logs/awslogs.log.

Is there any other piece I am missing? Looked through the official documentation with no luck so far.

Were you able to resolve this issue ? – Jalal El-Shaer Feb 05 '22 at 04:38 — Jalal El-Shaer, Feb 05 '22 at 04:38

score 0 · Answer 1 · edited Sep 14 '21 at 17:06

Instead of adding your own policy to the instance role, try adding the CloudWatchAgentServerPolicy managed policy.

I also removed the /etc/awslogs/awslogs.conf file definition when mine worked.

Finally, the agent only seems to create log groups in CloudWatch when lines are written to the log files. Make sure the files you are collecting are getting written to and see if the log groups get created. Good luck!

AWS Elastic Beanstalk: Custom Cloudwatch Logs not showing despite IAM permissions and custom config

1 Answers1