Stack Overflow
Stack Overflow

Questions tagged [amazon-cloudwatchlogs]

Amazon CloudWatch Logs allows aggregation and management of log events sent to it by AWS or custom services. Use this tag when asking about log groups, log streams, and metric filters. Use amazon-cloudwatch instead when asking only about CW metrics or CW alarms.

Amazon CloudWatch Logs allows aggregation and management of log events sent to it by AWS or custom services.

Usage Guidance

  • Use this tag when asking about log groups, log streams, metric filters, etc.
  • Do not use this tag when asking only about CW metrics or CW alarms; use instead.

Resources

862 questions
64
votes
5 answers

Amazon Cloudwatch Logs Insights with JSON fields

I am trying to use Logs Insights with data containing JSON in one of the fields, and to parse the JSON fields My data looks like the following when I put it in insights with the starter code fields @timestamp, @message | sort @timestamp desc | limit…
Cyril Duchon-Doris
  • 12,964
  • 9
  • 77
  • 164
57
votes
2 answers

Filter by timestamp query on AWS Cloudwatch Logs Insights

I am trying to use AWS Cloudwatch Logs insights in order to search in some quite old logs of our lambda functions. I am reading this guide on AWS docs, but nowhere is documented how you can filter by timestamp. I have tried the below: fields…
babis21
  • 1,515
  • 1
  • 16
  • 29
52
votes
2 answers

Cloudwatch Log Alert - How to include error / exception / stack trace data in email notification

I just configured Cloudwatch logs on my ec2 instances and am loving it so far. I also set up alerts for certain keywords, like "ERROR". While the email alert seems to be working fine, I was wondering if there's a way to fine-tune the alert email to…
Bhardwaj
  • 535
  • 1
  • 4
  • 6
44
votes
4 answers

Set expiration of CloudWatch Log Group for Lambda Function

By default when I create a Lambda function, the CloudWatch Log Group is set to Never Expire. Is it possible to set the expiration (saying 14 days) so I don't have to set it manually from the console after creation? Updated#1 Thanks to @jens…
niqui
  • 1,562
  • 1
  • 16
  • 28
43
votes
7 answers

My AWS Cloudwatch bill is huge. How do I work out which log stream is causing it?

I got a $1,200 invoice from Amazon for Cloudwatch services last month (specifically for 2 TB of log data ingestion in "AmazonCloudWatch PutLogEvents"), when I was expecting a few tens of dollars. I've logged into the Cloudwatch section of the AWS…
Sam Critchley
  • 3,388
  • 1
  • 25
  • 28
42
votes
2 answers

Amazon Cloudwatch Logs Insights parse with regex

I'm trying to extract an ephemeral field with the parse command. Unfortunately, the log format is such that the glob expression is not enough for it, thus I need to use regex. The regex itself is fine, but I just can't make the command to extract…
tnissi
  • 907
  • 1
  • 8
  • 12
40
votes
4 answers

Aggregate logs by field value and plot as multiple series using AWS CloudWatch Insights

Analysing some log files using AWS CloudWatch Insights, I can plot a count aggregated in time bins with: | stats count(*) by bin(1h) This produces a graph, as expected, aggregating all logs in each time bin. I want to split this data by a 'group'…
Tom Dufall
  • 871
  • 1
  • 10
  • 21
37
votes
1 answer

How can I sort results by aggregate in Cloudwatch Log Insights?

I have a pretty straightforward query: fields @timestamp, req.url, msg | sort @timestamp desc | filter msg = "request completed" | stats count() by req.url It presents all requests served by my app aggregated by url. However, I would also like to…
Max Yankov
  • 12,551
  • 12
  • 67
  • 135
34
votes
2 answers

How to search for plain text in cloudwatch logs insights?

I need to be able to search for certain errors using Cloudwatch logs insights. I tried to use parse syntax but failed to get any results. Sample error logs from my forever log StatusCodeError: 400 - "[{"errorCode":400002,"message":"Field Validation…
Mahela Wickramasekara
  • 603
  • 1
  • 11
  • 22
34
votes
5 answers

Count unique values in aws cloudwatch metric

I have a set of cloudwatch logs in json format that contain a username field. How can I write a cloudwatch metric query that counts the number of unique users per month?
postelrich
  • 3,274
  • 5
  • 38
  • 65
29
votes
3 answers

AWS Cloudwatch Filter and Pattern Syntax

I'm following the instructions here https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/FilterAndPatternSyntax.html but it's not working as i'm expecting it to. I currently have the following cloudwatch log subscription filter pattern: ?…
Ebrahim Moshaya
  • 727
  • 3
  • 11
  • 23
28
votes
2 answers

How do we sort CloudWatch stream logs by 'most recent' in AWS console?

Is there a way to filter CloudWatch log streams by 'most recent' to oldest within the AWS console? I am having to scroll incredibly far down to get to my most recent log messages. I have tried filtering by 30s - 5mins which works for now, I just…
Cody Mitchell
  • 431
  • 1
  • 6
  • 9
26
votes
5 answers

AWS Elastic Beanstalk: Add custom logs to CloudWatch?

How to add custom logs to CloudWatch? Defaults logs are sent but how to add a custom one? I already added a file like this: (in .ebextensions) files: "/opt/elasticbeanstalk/tasks/bundlelogs.d/applogs.conf" : mode: "000755" owner: root …
Joël
  • 1,563
  • 1
  • 18
  • 31
26
votes
2 answers

Search AWS CloudWatch Logs with special character in JSON property name

We use AWS CloudWatch Logs to debug our services running on AWS, and a number of the property names in our logs use colons as delimiters, but we can't figure out how to search using these property names. Here's an example of a log: { "Counts": { …
EmptyArsenal
  • 7,314
  • 4
  • 33
  • 56
25
votes
1 answer

How to run a group by in AWS Cloud Watch Logs Insights

I have CWL Entries as below. Showing entries in SQL Type for clarity Name City 1 Chicago 2 Wuhan 3 Chicago 4 Wuhan 5 Los Angeles Now I want to get below output City Count Chicago 2 Wuhan 2 Los Angeles …
Unbreakable
  • 7,776
  • 24
  • 90
  • 171
1
2 3
…
57 58