Alert or monitor, when your s3 bucket hits 90% of max limit for the particular account

Question

We recently hit the limit for max no of s3 buckets in our AWS account. We started to look for something which sends alerts when we use 90% s3 buckets quota in our account. But we not been able to find no such metric in cloud watch. Also, we looked for a similar rule in cloud custodian but no luck.

Expected result:

If S3 bucket account limit is 100 for an account.

If a number of buckets reach 90 we expect an alert to be sent "90% of buckets quota used".

Is this scenario possible?

score 2 · Accepted Answer · answered Aug 28 '19 at 13:54

You could set up a Cloudwatch Event to pick up (via CloudTrail calls) every time a bucket is created in S3, which you could then use to trigger a lambda function which counts the buckets, and then sends a notification to SNS to notify whoever or whatever to take action. In Python (boto3) this would look something like:

import boto3

s3 = boto3.client('s3')     
number_of_buckets = len(s3.list_buckets()['Buckets'])
if number_of_buckets >= 90:
   # send an alert via SNS

Instead of using a Cloudwatch Event from the API you could also trigger the lambda on a cron schedule (eg once a minute or daily etc)

Before asking the question here, we had implemented in the same way as your answer and after searching a lot it seems the only way to do it now. — Prashant Tiwari, Aug 30 '19 at 11:54

score 0 · Answer 2 · answered Aug 29 '19 at 17:19

There is a service called: Service Quotas. It was released on Jun 24, 2019. Apparently it can do what you want (can't confirm, didn't try it yet).

View AWS Service Quotas

The Service Quotas console provides quick access to the AWS default quota values for your account, across all commercial Regions. When you select a service in the Service Quotas console, you'll see the quotas and whether the quota is adjustable. Applied quotas are overrides, or increases for a particular quota, over the AWS default value.

Request a Service Quota Increase

For any adjustable service quotas, you can use Service Quotas to request a quota increase. To request a quota increase, in the console simply select the service and the specific quota, and choose Request quota increase. You can also use the API or command line interface (CLI) tools to request service quota increases.

View Current Utilization

If your account has been active a while and a resource has been used, you can view a graph of your quota utilization.

Set Amazon CloudWatch Alarms for Approaching Quotas

For supported services, you can manage your quotas by configuring CloudWatch alarms to monitor usage and alert you to approaching quotas.

https://aws.amazon.com/about-aws/whats-new/2019/06/introducing-service-quotas-view-and-manage-quotas-for-aws-services-from-one-location/

User Guide:

https://docs.aws.amazon.com/servicequotas/latest/userguide/ServiceQuotasUserGuide.pdf

As mentioned in the description, there is no metric filter for s3 to support this feature in cloudwatch alarms. Thus, not possible via service quota. — Prashant Tiwari, Aug 30 '19 at 11:56

score 0 · Answer 3 · answered Sep 18 '19 at 06:05

Wouldn't you rather increase your S3 bucket limit? There is a useful account service limit policy that will request to raise your S3 bucket limit by 25% or whatever percent is defined in your policy.

policies:
  - name: account-service-limits
    resource: account
    filters:
      - type: service-limit
        services:
          - S3
        threshold: 90
    actions:
      - type: request-limit-increase
        percent-increase: 25

Alert or monitor, when your s3 bucket hits 90% of max limit for the particular account

3 Answers3