1

I'd like to setup some monitoring for capturing access denied and unauthorized operation errors in my AWS account. I'd like to capture all of these events across different AWS services but have run into some issues. I've initially setup some a couple of cloud watch rules that trigger a basic lambda function but I'm not capturing the events I'm looking for. Below are a couple of the rules events that trigger a Lambda function (ideally I'd filter these down from capturing all events once I get this working).

{
  "source": ["aws.cloudtrail"],
  "detail-type": ["AWS API Call via CloudTrail"],
  "detail": {
    "eventSource": ["cloudtrail.amazonaws.com"]
  }
}
// All cloud watch
{
  "source": [
    "aws.cloudwatch"
  ]
 }

I've been testing with the AWS CLI, making calls to resources I don't have access too e.g. :

  • aws dynamodb list-tables
  • aws ec2 describe-instances
  • List item

I'm getting the correct errors in response from the CLI but I'm not seeing these events being propagated to the Lambda (I am seeing other non errors triggering the Lambda though).

Do I have to configure this more narrowly or am I missing a step for collecting these errors?

Thanks for the help

MattLock
  • 174
  • 3
  • 17

1 Answers1

2

aws dynamodb list-tables, aws ec2 describe-instances and List item do not work because List*, Get* and Describe* API events are not supported by CloudWatch Events:

enter image description here

If you want to capture such events, you have to do it through CloudWatch Logs. Namely, setup your CT trail to push events to CloudWatch Logs. Then, setup a subscription filter to your lambda. This will push all API events to your lambda through CloudWatch Logs. Subsequently, you will be able to detect AccessDenied errors in the events and perform other actions you require.

Other possibility would be to setup metric filter on the logs so that AccessDenied errors are captured to create a CloudWatch metric. Then you setup a CloudWatch Alarm based on the metric. This will trigger an alarm, but it will not pass the details of event that triggered the alarm to your function. You will only know that access denied happened, but without further details.

Update

create-table works as expected. I used the following rule in my tests which I hooked up to SQS for quick checks:

{
  "detail-type": [
    "AWS API Call via CloudTrail"
  ],
  "detail": {
    "eventSource": [
      "dynamodb.amazonaws.com"
    ],
    "errorCode": [
      "AccessDenied"
    ]
  }
}
Marcin
  • 215,873
  • 14
  • 235
  • 294
  • Thank you for the detailed response @Marcin! I didn't realize that constraint with Cloudwatch events. I did try using another api call 'aws dynamodb create-table' and had similar results though, I'm leaning towards there is likely something wrong with my setup. Do you know if there are any other restrictions or if a 'create' doesn't qualify as a 'write' API call? – MattLock Jun 02 '21 at 00:38
  • 1
    @MattLock No problem. I updated the answer. create table works with events. – Marcin Jun 02 '21 at 00:49
  • Thanks for the help @Marcin, I'm going to accept the answer / reward the bounty. If I wanted to make that more broad and catch all cloud trail AccessDenieds, could I replace the event source with '"eventSource": ["cloudtrail.amazonaws.com"]' ? – MattLock Jun 02 '21 at 12:32
  • 1
    @MattLock Thanks. This would only limit your rule to cloudtrial API events, not other services. To apply it to all services you can try removing `eventSource` all together. But I haven't tested it. – Marcin Jun 02 '21 at 12:49
  • 1
    Incase you were interested, removing the eventSource didn't work unfortunately – MattLock Jun 02 '21 at 23:58
  • 1
    @MattLock Maybe [this](https://stackoverflow.com/questions/62406933/aws-eventbridge-pattern-to-capture-all-events/62407802#62407802) will be helpful. – Marcin Jun 03 '21 at 00:59
  • Thanks for the link, I wasn't able to set that as a CloudWatch rule as the syntax in that post was 'invalid' but I was able to set the source as an EventBridge rule. I didn't realize there was a difference between what type of filtering you could do on both services. – MattLock Jun 03 '21 at 13:51