15

I have just started using AWS serverless using SAM and have run into the problem below:

SAM invoke can't seem to find a python module that my lambda handler is importing but I can otherwise import that module.

The structure of my project is:

root-dir/

  • mymodulename/
    • mymodule.py
  • aws/
    • sam/
      • template.yaml
      • packaged.yaml
      • myawsservice/
        • app.py
      • test/
        • test_event.json

I am working in a python3.6 virtual environment and I have installed my python package (mymodulename) using setup tool:

python setup.py develop

So if I do a pip freeze in my virtual environment I see:

Pillow==6.0.0
PyPDF2==1.26.0
reportlab==3.5.20
mymodulename==0.5.0

Also if I go:

cd root-dir/aws/sam/
python

>>> import mymodulename

This succeeds. So it is on the python path.

But when I invoke sam locally i.e.:

sam local invoke MyAWSServiceFunction --event ../sam/test/test_event.json

I get the error message:

2019-05-02 09:19:17 Found credentials in shared credentials file: ~/.aws/credentials
2019-05-02 09:19:18 Invoking app.lambda_handler (python3.6)

Fetching lambci/lambda:python3.6 Docker container image......
2019-05-02 09:19:19 Mounting /home/myname/root-dir/aws/sam/myawsservice as /var/task:ro,delegated inside runtime container
START RequestId: 245daefe-ecfb-4530-9d15-cf07f55e0f3d Version: $LATEST
Unable to import module 'app': No module named 'mymodulename'
END RequestId: 245daefe-ecfb-4530-9d15-cf07f55e0f3d
REPORT RequestId: 245daefe-ecfb-4530-9d15-cf07f55e0f3d Duration: 31 ms 
Billed Duration: 100 ms Memory Size: 128 MB Max Memory Used: 19 MB

{"errorMessage": "Unable to import module 'app'"}

Here is the code for my app.py:

from mymodulename.mymodule import mymodulefunction

def lambda_handler(event, context):    
    return mymodulefunction(event['body'])

I can run the code above from my unit test just not from the sam invoke environment.

template.yaml:

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: > something

Globals:
  Function:
    Timeout: 3

Resources:
  MyAWSServiceFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: myawsservice/
      Handler: app.lambda_handler
      Runtime: python3.6

Help appreciated.

EDIT:

One workaround is to put the app.py, requirements.txt and template.yaml in the root directory and do:

sam build

This will store the 3rd party dependencies and my own python module together and template.py will now be able to find it during sam local invoke.

This is however not practical for bigger projects, and I guess the answer for now might be something like this. Although this is for a serverless.yaml. I'm not sure if SAM supports this syntax. Guessing no.

Willeman
  • 720
  • 10
  • 24
  • Have you tried using lambda layers? You can deploy 3rd party libs along with any other libs you create to a lambda layer and import that layer into your lambda function. https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html – rhn89 Apr 01 '20 at 04:58
  • Take a look if this post could help https://stackoverflow.com/a/76949855/2153237 – Jose Carlos Ramos Carmenates Aug 22 '23 at 03:10

1 Answers1

6

It's being a long time since this question was opened, but it might help to someone. How I fixed it:

  • Move your code from mymodulename/ to mymodulename/mymodulename/__init__.py
  • In mymodulename create a setup.py who will be responsible for creating a package called mymodulename
  • In your requirements.txt of your lambda function add this: -e mymodulename/

The -e references to your local code, this link might helps also.

jtlz2
  • 7,700
  • 9
  • 64
  • 114
Lücks
  • 3,806
  • 2
  • 40
  • 54
  • could you share your setup.py? I'm facing this same issue – Raul Quinzani Jun 08 '21 at 03:13
  • from setuptools import setup setup( name='mymodulename', version='1.0', description='A useful module', author='Your name', author_email='example@example.com', packages=['mymodulename'], #same as name py_modules=['mymodulename'], # the py file name of your methods, like mymodulename/mymodulename/mymodulename.py install_requires=['pymongo', 'dnspython'], #external packages as dependencies ) – Leslie Wong Jul 01 '22 at 20:43
  • One more note: the function you define within mymodulename.py should be imported into __init__.py, like ' from .mymodulename import myfunction ' – Leslie Wong Jul 01 '22 at 20:48
  • This would definitely help. https://github.com/aws/aws-lambda-builders/blob/develop/tests/integration/workflows/python_pip/testdata/local-dependencies/setup.py – Leslie Wong Jul 01 '22 at 21:43