Finding an equivalent of environment variables in AWS Lambda Layers?

Question

I am writing a serverless app on AWS.

I have broken up the app into many CloudFormation stacks. I am using CDK (in Python) to create the CF stacks to deploy the app.

A core requirement of my lambda functions, of course, is the ability to log events. To handle this (and all message passing in the app), I have created a custom EventBridge bus in one of my stacks. The name of the event bus is an output of the stack.

Because the logging functionality will be common to many lambda functions, it seems appropriate to put the logging functionality into a lambda layer. I can then have all of my lambda functions implement that layer, and logging will be automagically available to all of my lambda functions.

The problem is that my logging code needs to know which EventBridge event bus to write events to. I don't want to hardcode this, because I may deploy a 'dev' and/or 'test' and/or 'prod' version of the stack simultaneously and the logging layer in each environment needs to log to its environment's logging bus.

Because a lambda layer runs in the context of the parent lambda function, I can't set an environment variable on the layer.

I could set a 'LoggingEventBus' environment variable on all the lambda functions that use this layer. But that seems repetitive and error prone. (Although it's the best solution I can come up with at the moment.)

I could store the event bus in an SMS parameter store. But good Lord, lookup up that parameter every time any function in my app wants to log anything would just be ridiculous. (And I'm back to having to figure out a way for the logger to determine if it's in dev/test/prod to look up the right bus.)

I'm actually considering some build process during 'cdk synth' that modifies the source code of the logging layer and does a string replacement of the event bus name, so that the name actually is hardcoded by the time the code is deployed. But that solution has all kinds of danger signals around it.

Ideally, there would be some kind of 'environment variable' for the layer itself. But no such thing exists, and I acknowledge that such a feature would not be compatible with how most language runtimes work.

How have others solved this? Is there a 'right' answer for putting a setting in a lambda layer?

Did you find any solution for this ? – Deeraj Theepshi Jun 23 '21 at 01:00 — Deeraj Theepshi, Jun 23 '21 at 01:00

score 5 · Answer 1 · answered Feb 18 '21 at 07:01

You are right that there is no concept of environment variables for Lambda Layers.

I could set a 'LoggingEventBus' environment variable on all the lambda functions that use this layer. But that seems repetitive and error prone.

We also use Layers for some logic (involving SNS Topics etc) which is common to multiple Lambda functions, and we do it very similar to what you suggested above.

Only difference is that we just set the stage in environment variables, not the values of our SNS Topics (or in your case the EventBridge name). And then let the Layer decide which SNS Topic to use based on the stage set in env.

In our Layer code, we use a config like this -

{
  "Prod": {
    "SNSTopicARN": "ProdARN",
    "OtherConfig": "ProdValue"
  },
  "Dev": {
    "SNSTopicARN": "DevARN",
    "OtherConfig": "DevValue"
  }
}

Then in the Layer Code, we have

config = configJson.get(env.STAGE)

Not the error-prone I guess :)

And we can add any more number of config variables in the Layer without touching the Lambda functions deployment.

Finding an equivalent of environment variables in AWS Lambda Layers?

1 Answers1

Linked