Get overview of cold starts in AWS Lambda

Question

Is it possible to get an overview of AWS Lambda cold starts, i.e. how many there were and how long they took?

I know there are solutions for cold starts, and I know I could log something on a cold start, but that is not what I need. I want an overview of cold starts in the past.

I know that a new log stream is created when a cold start occurs, so I though I'd use CloudWatch Insights to get the first record of type REPORT for every log stream. But I can't manage to write a query that would group the reports by log stream and then take the first for every group.

This would give me an overview that I might be able to analyze in Excel, but the data is just too large for Excel:

filter @type = "REPORT"
| fields @logStream, @duration, @timestamp
| sort @logStream, @timestamp asc

I can get the stats max(@duration) by @logStream but that's not what I want. I want the first record and the take the @duration from there. Is this at all possible? Or should I be looking elsewhere?

what do you mean by overview? what specific data are you are after? — Nath, Jul 11 '19 at 14:03

score 3 · Answer 1 · answered Jul 16 '19 at 06:43

I eventually found a way. Because AWS lambda creates a new log stream for every lambda instance it creates, we can look at the first REPORT in each log stream we're interested in.

The idea is to perform the following steps:

1. get the set of log streams you want

In my case, I needed 2 months. Because the log streams have a name that starts with / (e.g. 2019/05), I could query AWS for that:

await cloudWatchLogs.describeLogStreams({
    logGroupName: "/aws/lambda/my-lambda-name",
    logStreamNamePrefix: "2019/05",
    descending: true
}).promise();

2. get the log events

For each log stream, I can now get the log events:

await cloudWatchLogs.getLogEvents({
    logGroupName: "/aws/lambda/my-lambda-name",
    logStreamName: logStream.logStreamName,
    startFromHead: true
}).promise();

3. find the first REPORT

Then we run over the events and find the first REPORT message. This contains the duration of this invocation. As this is the first one in the log stream, it's the duration of our cold start.

for (let event of events.events) {
    if (event.message.startsWith("REPORT")) {
        const duration = /Duration:(.*)Billed/.exec(event.message);
        if (duration) {
            console.log(`${logStream.logStreamName}  --  ${duration[1]}`);
        }

        break;
    }
}

I also created a Gist with the full code.

score 3 · Accepted Answer · answered Sep 19 '20 at 17:24

Not sure when it was added, but CloudWatch Logs now details the duration of the cold-start in the Init Duration field: "For the first request served, the amount of time it took the runtime to load the function and run code outside of the handler method." If Init Duration isn't included, no cold-start was triggered, and your invocation was handled by an instance that was initialized / running earlier.

Here's the documentation on that for the Node.js runtime.

Get overview of cold starts in AWS Lambda

2 Answers2

1. get the set of log streams you want

2. get the log events

3. find the first REPORT