8

I'm trying to create elasticsearch cluster using terraform, But i'm getting this error

11:58:07 * aws_cloudwatch_log_resource_policy.elasticsearch-log-publishing-policy: Writing CloudWatch log resource policy failed: LimitExceededException: Resource limit exceeded.
11:58:07 * aws_elasticsearch_domain.es2: 1 error(s) occurred:

I initially thought that this resource limit error is unable to create log groups. But when i raised a Ticket with AWS team , they said there is "no throttling on CreateLogGroup API for this account in IAD"

ElasticSearch has about 10 clusters running. I'm not sure which resource limit has exceeded.

Can someone pls explain me the above error.

Update:

data "aws_iam_policy_document" "elasticsearch-log-publishing-policy" {
  statement {
    actions = [
      "logs:CreateLogStream",
      "logs:PutLogEvents",
      "logs:PutLogEventsBatch",
    ]

    resources = ["arn:aws:logs:*"]

    principals {
      identifiers = ["es.amazonaws.com"]
      type        = "Service"
    }
  }
}

resource "aws_cloudwatch_log_resource_policy" "elasticsearch-log-publishing-policy" {
  policy_document = "${data.aws_iam_policy_document.elasticsearch-log-publishing-policy.json}"
  policy_name     = "elasticsearch-log-publishing-policy"
}

I tried to apply this using terraform target, i think the error is here, does AWS have a limit on number of custom policies we create, I could not find an option to request an increase.

user6826691
  • 1,813
  • 9
  • 37
  • 74
  • Did you check the CloudTrail logs for the failed API-Call? Then you'll know which one actually failed. – Maurice Jan 07 '21 at 15:46

2 Answers2

9

does AWS have a limit on number of custom policies we create, I could not find an option to request an increase.

Yes, the limit can't be change and it is:

Up to 10 CloudWatch Logs resource policies per Region per account. This quota can't be changed.

Marcin
  • 215,873
  • 14
  • 235
  • 294
  • How do i use an existing policy using terraform to create the log group, there is an option in console but i could not figure how do it in terraform. – user6826691 Jan 08 '21 at 21:28
  • @user6826691 You would have to import it to TF so that TF can manage it, and then update it. – Marcin Jan 08 '21 at 21:53
4

Up to 10 CloudWatch Logs resource policies per Region per account. This quota can't be changed.

To resolve this error please delete the resource policies which are of no more use to you. Command to list and delete the resource policies

aws logs describe-resource-policies

aws logs delete-resource-policy --policy-name PolicyNameToBeDeleted

YOGESH CHANDRAKANTH
  • 89
  • 2