15

Hi I have my AWS Lambda and I wanted to add a layer to it. I would like to be able to just test single methods of lambda. However many of them use layer logic and because of that it doesn't seem to me to be easy. What is the best approach to do this ?

One approach would be to package layer, host is somewhere and use it as dependency. In that case why even bother to use layers ?

The other idea I have is to deploy lambda locally with sam-cli. I know how to use it to test the whole lambda logic but I can't see how to unit test methods separately ;/ What are your experiences ? KR

EDIT. My solution

  • Add pytest

  • Place all the tests in test directory

  • Add test lambda handler which invokes tests

import pytest def lambda_handler(event, _): res = pytest.main(['-x', './tests']) return res

  • Add template.yml which points to previously created lambda handler

Resources: MyFunction: Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction Properties: CodeUri: src/ Handler: test.lambda_handler Runtime: python3.6 Events: MyInfo: Type: Api # More info about API Event Source: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#api Properties: Path: /my-service/test Method: get Environment: Variables: ELASTICSEARCH_DOC_TYPE: "article" ELASTICSEARCH_INDEX: "artilces" ELASTICSEARCH_HOST: "elastic" ELASTICSEARCH_PORT: "9200" ELASTICSEARCH_URL: "http://my_elastic.com:9200" Layers: - arn:aws:lambda:eu-west-1:XXXXXXXXXXXXX:layer:lambda_layer:37

  • Run sam local invoke --no-event
Clyde Barrow
  • 1,924
  • 8
  • 30
  • 60

3 Answers3

5

For my case, I was using the moto library to mock AWS Services so using sam local was not an option. I just added the path of my lambda layer to my sys path in my test files and it did the trick for me.

Ghamgui Khaled
  • 364
  • 2
  • 7
  • Hey just stumbled upon this and this is exactly the situation I'm in. Do you mind expanding a bit on what you did? – ConorBaumgart Jun 30 '21 at 13:55
  • 1
    Sure thing, I have first defined a function that append to my python path my layer directory and also the folder containing all of my lambda function. All I need to do is call this function at the top of my unittest test class and then simply refer to the functions to run my unit/integration tests. – Ghamgui Khaled Jun 30 '21 at 17:24
  • So you basically added a conftest.py file that contains a function that appends your layer directory to your python path. And then within each test file you call that function at the top of the file before the imports? – ConorBaumgart Jun 30 '21 at 19:45
  • 1
    That's exactly what I did – Ghamgui Khaled Jul 01 '21 at 11:04
  • @GhamguiKhaled Do you have any insights on how to handle requirements for layer and lambda function ? My custom layer has several dependencies (say defined in requirements.txt). My lambda function also has other dependencies. Now when I run unit test cases both dependencies should be available in the virtual environment. – Sujith C P Oct 28 '21 at 11:47
  • @SujithCP, I personally didn't use a Venv for the dependencies. In my case I only had my dependencies in one layer and they were shared between my lambda function. So pip install did the trick. It might be problematic if the lambda functions were using different versions of certain package. I think you should use in that case a venv. Hope it answers your question – Ghamgui Khaled Nov 06 '21 at 11:08
1

I was trying to do the same thing and couldn't find an answer anywhere myself. After thinking about the problem for a bit, I came up with what I think is a pretty clean approach. I'm finishing a blog post about it which I will post here but basically I used a Factory pattern to load my layer code and mocked out the code during unit tests.

It would go something along these lines:

Assume all of your layer code is encapsulated in a Class called SharedCode. You can use a static factory that loads the SharedCode using a conditional import:

class Factory:

   def __init__(self):
       self._shared = None

   def set_shared_code(self, shared_code):
       self._shared = shared_code

   def get_shared_code(self):
       if not self._shared:
           from shared_code import SharedCode
           self._shared = SharedCode()

       return self._shared


FACTORY = Factory()

In your tests, you can use the Factory setter to mock out the Layer code but in your actual Lambda function, you simply load the shared code using the Factory which thanks to the conditional import and the Lambda runtime that injects the Layer it will be able to load it with no problem.

Hope this helps and I'll circle back to have a more complete example once I finish creating the samples for my blog post.

EDIT: The blog post I mentioned above is ready and you can read it here:

Abraham Sultan
  • 11
  • 2
  • So in your approach I must write a mock for each method I use in the lamda - it might be a little pain in the neck - nice solution but I hope the AWS will come up with some idea of easy testing using the layer logic – Clyde Barrow Apr 28 '19 at 10:15
  • Hi Clyde, yes that is accurate, however that is also a normal common pattern when unit testing since you want to focus the test in the current class and is normal to mock out dependencies. – Abraham Sultan May 01 '19 at 02:11
  • The blog post link is dead – Beefster Jul 15 '21 at 22:40
  • Hey Beefster unfortunately our site was changed and we lost the original references but here is the new link: https://nuvalence.io/blog/code-reuse-and-serverless-apps?rq=code%20reuse%20and%20serverless%20apps From what it looks like the code visualizer broke as well but you should still be able to make up the solution. Hope it helps. Abe – Abraham Sultan Jul 17 '21 at 22:10
0

Lambda layer works as a Python package too in the Python runner.

Your Jenkins pipeline of the Lambda layer could generate and/or publish both a Lambda layer and a Python package.

And then your unit tests of your Lambda function could call the Python package, while the actual execution of your Lambda function will call the Lambda layer.

Dayong
  • 5,614
  • 1
  • 14
  • 6