Import Python module into AWS Lambda

Question

I have followed all the steps in the documentation: https://docs.aws.amazon.com/lambda/latest/dg/lambda-python-how-to-create-deployment-package.html

1. create a directory.

2. Save all of your Python source files (the .py files) at the root level of this directory.

3. Install any libraries using pip at the root level of the directory.

4. Zip the content of the project-dir directory)

But after I uploaded the zip-file to lambda function, I got the error message when I test the script

my code:

    import psycopg2
    #my code...

the error:

    Unable to import module 'myfilemane': No module named 'psycopg2._psycopg'

I don't know where is the suffix '_psycopg' from...

Any help regarding this?

Answer 1

You are using native libraries with lambda. We had this similar problem and here is how we solved it.

Spin a machine with AWS supported AMI that runs your real lambda.

https://docs.aws.amazon.com/lambda/latest/dg/current-supported-versions.html

As this writing, it is,

AMI name: amzn-ami-hvm-2017.03.1.20170812-x86_64-gp2

Full documentation in installing native modules your python lambda.

https://docs.aws.amazon.com/lambda/latest/dg/lambda-python-how-to-create-deployment-package.html

Install the required modules required for your lambda,

pip install module-name -t /path/to/project-dir

and prepare your package to upload along with the native modules under lambda ami environment.

Hope this helps.

Answer 2

I believe this is caused because psycopg2 needs to be build an compiled with statically linked libraries for Linux. Please reference Using psycopg2 with Lambda to Update Redshift (Python) for more details on this issue. Another [reference][1] of problems of compiling psycopg2 on OSX.

There are a few solutions, but basically it comes down to installing the library on a Linux machine and using that as the Psycopg2 Library in your upload package.