lambda python function with external library psycopg2 - No module named psycopg2

Question

Error Message - No module named psycopg2
File used in zip - https://github.com/jkehler/awslambda-psycopg2
Code Snippet -

    #!/usr/bin/python
import psycopg2
import sys
import pprint
import datetime

def lambda_handler(event, context):

#Connect to RedShift
conn_string = "dbname='XXXX' port='5439' user='XXX' password='XXXX' host='XXXXXXXXXXXX'";

conn = psycopg2.connect(conn_string);
cursor = conn.cursor();

cursor.execute("begin transaction");
cursor.execute("truncate table XXXX");
cursor.execute("truncate table XXXX");
cursor.execute("truncate table XXXX");
cursor.execute("delete from XXXX");
cursor.execute("insert into XXXX");
cursor.execute("truncate table XXXX");
cursor.execute("truncate table XXXX");
cursor.execute("truncate table XXXX");
cursor.execute("end transaction");

conn.commit();
conn.close();

Extracted and Copied psycopg2 in windows into my AWS Lambda zip package along-with my python file and site packages.
Did I miss anything?

EDIT
Recreated the package with zipping the file on Amazon Linux. Still same error.

That's not valid python code. Also: if you want to "bundle" together packages you should **freeze** your application instead of manually copying files. — Bakuriu, Sep 15 '16 at 15:13
@Bakuriu - Can you please elaborate where did I mess up in my python code? I am new to python and this code executes on my windows machine. — Rahul Gupta, Sep 15 '16 at 15:37
Perhaps he's referring to the lack of indentation, which is significant in Python. — Jeff Learman, Sep 15 '16 at 17:47
@JeffLearman - Maybe, but I do have indentation in my source code. — Rahul Gupta, Sep 15 '16 at 17:53

score 1 · Answer 1 · answered Sep 15 '16 at 02:42

1

psycopg2 is a compiled module. Copying the Windows version won't work because Lambda runs on top of Amazon Linux. According to the docs it is possible to run native executables and libraries on lambda, but you'll need to find a way to build psycopg2 for that platform and statically link all its libraries or bundle the dynamic libs. This is likely to be challenging unless you're familiar with the C and Python toolchain or someone else has already done it for you.

answered Sep 15 '16 at 02:42

Paul Kehrer

13,466
4
40
57

I tried with zipping the file on Amazon Linux, and still I get the same error. From the github link I posted in question, psycopg2 has already been built I just had to add it to my directory. – Rahul Gupta Sep 15 '16 at 14:52

score 0 · Answer 2 · answered Mar 01 '22 at 08:34

AWS lambda doesn't have all the Python libraries that are in PIP, it is the minimalist amazons python to improve the execution time. So if you want to use other libraries you need to download the source version of them.

You can find them in pypi.org

Here is the reference document that might help you. How to create aws lambda layers.

This page shows the detailed step by step on how to create layers.

lambda python function with external library psycopg2 - No module named psycopg2

2 Answers2