Filter by timestamp query on AWS Cloudwatch Logs Insights

Question

I am trying to use AWS Cloudwatch Logs insights in order to search in some quite old logs of our lambda functions. I am reading this guide on AWS docs, but nowhere is documented how you can filter by timestamp. I have tried the below:

fields @timestamp, @message
| filter @timestamp > '2019-12-04T18:09:10.000+01:00'
| limit 200
| sort @timestamp desc

but doesn't work (returns 0 results).

Initially, I was trying to find out if there is a way to sort the log groups by a timestamp column (instead of the default which is the log group name), when I came across this feature request since 2015 - this is not resolved in eu-west-1 and they suggest to use the new log insights, but I can't make this work.

Does anyone know how I can filter logs by timestamp, or if this is even possible with Cloudwatch logs insights?

Thanks!

I have the same error, I need to know too – lambda Mar 16 '20 at 16:56 — lambda, Mar 16 '20 at 16:56

score 75 · Accepted Answer · answered Mar 17 '20 at 13:07

75

Filtering on timestamp is done with the range selector on the top right in the Logs Insights Console or with the startTime and endTime parameters on the StartQuery API.

You could do further filtering using timestamp values in millis (see below for an example), but the overall range still needs to be wider than what you're using in the query itself.

fields @timestamp, @message
| fields tomillis(@timestamp) as millis
| filter millis > 1578182400000  # Sunday, 5. January 2020 0:00:00
     and millis < 1578268800000  # Sunday, 6. January 2020 0:00:00

answered Mar 17 '20 at 13:07

Dejan Peretin

10,891
1
45
54

1

OMG I didn't even see this filter on the top right! Thanks! I have also tried the millis filter, and it works. I didn't see this anywhere in AWS docs. Anyway, your answer helped a lot, thank you! – babis21 Mar 18 '20 at 07:52
4

`date -u '+%s%3N' -d 'TZ="UTC" 2020-01-05 00:00:00.000'`. Just change `TZ` for your timezone specific time – GypsyCosmonaut Dec 24 '20 at 18:00
Sometimes you will still not see the expected output. You can increase the number of output lines to 10,000 by simply adding `| limit 10000`. – alexrogo Aug 02 '21 at 13:40
If you're using the [Incite](https://github.com/gogama/incite) library for Go to interact with CWL Insights, use the `Start` and `End` fields in the [`QuerySpec`](https://pkg.go.dev/github.com/gogama/incite#QuerySpec) structure. – 0xbe5077ed Sep 16 '21 at 17:15

score 5 · Answer 2 · answered Aug 26 '22 at 16:34

5

You can use @timestamp it just expects a UNIX timestamp in seconds, instead of milliseconds, from epoch.

Like this @timestamp > 1575479350

See here https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL_QuerySyntax.html.

Although the docs don't mention you can do this I tested it out and it works.

Regarding the ISO string that you're trying to use there is a note in docs that...

answered Aug 26 '22 at 16:34

shmuels

1,039
1
9
22

thanks for the hint, however milliseconds epoch values worked for me, not seconds, also checked with `fields bin(1s) as bucket | filter (bucket=1678876961000)` – Ivan Samygin Mar 16 '23 at 12:13

Filter by timestamp query on AWS Cloudwatch Logs Insights

2 Answers2

Linked