22

AWS CloudWatch has Log Groups and Log streams. A log group seems reasonable to me: Each product (e.g. each Lambda function, each Sagemaker endpoint) has its own log group.

But then there are log streams. When does AWS CloudWatch create new log streams? Can I search all log streams of a log group?

John Rotenstein
  • 241,921
  • 22
  • 380
  • 470
Martin Thoma
  • 124,992
  • 159
  • 614
  • 958

1 Answers1

18

From the AWS Cloudwatch documentation you can see that a log stream is created each time the logs come from a different event source. In case of Lambda, it's one stream per Lambda container where each container might process multiple events.

A log stream is a sequence of log events that share the same source. Each separate source of logs into CloudWatch Logs makes up a separate log stream.

Yes, you can search all log streams of a log group using the CloudWatch Logs API. The FilterLogEvents action allows you to search through a log group.

rvelaz
  • 553
  • 7
  • 14
  • 10
    "In case of Lambda, each new request represents a new event source, thus a new log stream is created." Not accurate: it's one stream per Lambda container where each container might process multiple events. – Dunedan Feb 07 '18 at 08:58
  • Thanks @Dunedan I've amended the answer – rvelaz Feb 07 '18 at 15:06
  • 2
    What exactly is a lambda container? Is it the same as a lambda function? I have the impression, every time I click on "test" it creates a new log stream. But it definitely does not create a new log stream each time the lambda function is invoked. – Martin Thoma Feb 08 '18 at 08:33
  • 2
    Lambda functions are executed in containers. Concurrent requests of the same lambda function will be executed in the same container. When there are no more requests to execute a lambda function, the container will be terminated. – rvelaz Feb 13 '18 at 16:33
  • Did you know how many time to generate log of CloudWatch Log? i think sometimes the log generates delay but don't know exactly the range time of it. – Minh Loc Dec 10 '19 at 02:45
  • 1
    Concurrent means "at the same time". Concurrent requests of the same lambda function will be executed in **different** containers, not the same container at all. Each container handles exactly one request at a time. **Sequential** requests *might* be handled by the same container. https://aws.amazon.com/blogs/compute/container-reuse-in-lambda/ – Mark B Jan 22 '23 at 00:29