Partially informed by this answer to a similar question, and by trial/error and who knows what other googling, I ended up pivoting away from AWS Metrics and AWS Alarms, and instead utilized a new Lambda function that is triggered by any number of my select Log groups. I kept the SNS functionality, which is very easy to set up if you own your emailing domain.
Upon creation of a new Lambda function, the option to search for blueprints is presented, and searching log yields (among other options) the cloudwatch-logs-process-data blueprint. It can also be created from scratch, as its code is merely this:
const zlib = require('zlib');
exports.handler = async (event, context) => {
const payload = Buffer.from(event.awslogs.data, 'base64');
const parsed = JSON.parse(zlib.gunzipSync(payload).toString('utf8'));
console.log('Decoded payload:', JSON.stringify(parsed));
return `Successfully processed ${parsed.logEvents.length} log events.`;
};
I then chose for it as multiple Triggers the Log groups I wish to monitor, setting their Filter pattern to the word ERROR, and setting their Filter name to my desired email subject.
Using the console log from the blueprint, I created a test event of this format:
{
"awslogs": {
"data": "LONG_STRING_OF_ENCODED_CHARACTERS_FROM_ABOVE_CONSOLE_LOG_HERE"
}
}
And I modified the blueprint code more to my liking as this:
const zlib = require('zlib');
const aws = require('aws-sdk');
const ses = new aws.SES({ region: '<some-region-1>' });
exports.handler = async function (event) {
const payload = Buffer.from(event.awslogs.data, 'base64');
const parsed = JSON.parse(zlib.gunzipSync(payload).toString('utf8'));
const bodyText = JSON.stringify(parsed, null, 2).replace(/\\n/g, ' \\\n ').replace(/\\t/g, '\\\n ');
console.log('bodyText:', bodyText);
var params = {
Source: '<some_email@example.com>',
Destination: {
ToAddresses: ['<some_email@example.com>'],
},
Message: {
Subject: { Data: parsed.subscriptionFilters[0] },
Body: {
Text: { Data: bodyText },
},
},
};
return ses.sendEmail(params).promise();
};
And it works, I get an immediate email if a Lambda function logs an error (with no noticeable increase to my monthly AWS bill as well).
