7

I'm running an AWS Lambda script with a Python 2.7 runtime. However, whenever it initializes it begins to import the grequests library but fails because of it's dependency on gevent:

Gevent is required for grequests.

It seems it is successfully finding the grequests library (since it knows it needs gevent) but fails.

What I've tried so far:

pip install --ignore-installed grequests -t .

pip install --ignore-installed grequests -t ./lib

pip install --ignore-installed gevent -t .

pip install --ignore-installed gevent -t ./lib

And then I compress the contents of the directory and upload to AWS per the instructions here: http://docs.aws.amazon.com/lambda/latest/dg/lambda-python-how-to-create-deployment-package.html

It seems no matter what I try the Lambda is unable to locate gevent, but it's able to find other non-default libraries with no issue.

Tyler Mills
  • 353
  • 1
  • 3
  • 15
  • Where are you calling `pip install`? In your lambda script? – helloV Mar 27 '16 at 06:37
  • I'll add this to the original post, but I'm calling this from the terminal and zipping the contents for upload to AWS following these instructions: http://docs.aws.amazon.com/lambda/latest/dg/lambda-python-how-to-create-deployment-package.html – Tyler Mills Mar 27 '16 at 06:38

2 Answers2

3

As Gevent is based on libev it is most likely compiling binaries when being installed via pip.

You need to make sure that you are deploying binaries that are compiled for Amazon Linux if you want them to be executable in AWS Lambda. You can do so by building your deploy package on an EC2 instance that is running Amazon Linux.

Also check out this answer and this tutorial.

Community
  • 1
  • 1
birnbaum
  • 4,718
  • 28
  • 37
  • Thanks for this. I did try making this work by spinning up an EC2 instance and installing there and then including the result in my repo. However, I have the same issue. – Tyler Mills Apr 25 '16 at 18:14
  • Do you have the same issue when running the code on EC2? If yes, I would recommend to debug the problem there, because it should behave the same as in Lambda. Debugging stuff like this directly in Lambda is a real pain. – birnbaum Apr 25 '16 at 20:06
  • Fun fact, this is also the solution for getting the `lxml` library to run on Lambda. So if anyone is searching, just compile on the server. – Tyler Mills Feb 16 '17 at 21:44
3

I had to build gevent from src on an Amazon Linux instance. I put the resultant files in a zip if anyone needs them--just include them in your uploaded Lambda zip and you should be concurrent-ing like a boss.

https://github.com/brandonmp/aws-lambda-grequests

Brandon
  • 7,736
  • 9
  • 47
  • 72
  • It seems my issue was trying to compile the entire grequests on Amazon Linux which was running into an unrelated issue. Just compiling `gevent` works. Thanks! – Tyler Mills Jan 19 '17 at 16:46