1

Need to deploy lambda function in AWS using python dependencies like numpy, pandas. after making zip file of all and using with S3 path to execute found following error

Unable to import module 'lambda_function': Importing the multiarray numpy extension module failed. Most likely you are trying to import a failed build of numpy. If you're working with a numpy git repo, try git clean -xdf (removes all files not under version control). Otherwise reinstall numpy.

Original error was: cannot import name multiarray

gaurav pande
  • 23
  • 1
  • 7

1 Answers1

2

After much effort, I found that I had to create my deployment package from within a python3.6 virtualenv, rather than directly from the host machine. I did the following within a Ubuntu 16.04 docker image. This assumes that you have python3.6, virtualenv and awscli already installed/configured, and that your lambda function code is in the ~/lambda_code directory:

1) cd ~ (We'll build the virtualenv in the home directory)

2) virtualenv venv --python=python3.6 (Create the virtual environment)

3) source venv/bin/activate (Activate the virtual environment)

4) pip install numpy

5) cp -r ~/venv/lib/python3.6/site-packages/* ~/lambda_code (Copy all installed packages into root level of lambda_code directory. This will include a few unnecessary files, but you can remove those yourself if needed)

6) cd ~/lambda_code

7) zip -r9 ~/package.zip . (Zip up the lambda package)

8) aws lambda update-function-code --function-name my_lambda_function --zip-file fileb://~/package.zip (Upload to AWS)

Your lambda function should now be able to import numpy with no problems.

If you want a more out-of-the-box solution, you could consider using serverless to deploy your lambda function. Before I found the above solution, I followed the guide here and was able to run numpy successfully in a python3.6 lambda function.

cailan s
  • 121
  • 1
  • 4
  • Thanks cailan s its works for me but now i am facing problem to deploy sklearn in aws. if you have any idea your help is great ful for me – gaurav pande Mar 09 '18 at 06:31
  • I haven't used sklearn myself but [this guide](https://serverlesscode.com/post/scikitlearn-with-amazon-linux-container/) could be a starting point. If you post what exactly your problem is maybe I or someone else would be able to help – cailan s Mar 09 '18 at 23:16
  • This made my day at work easy. Great writeup. Be sure y'all have the bin in your path for whatever python you're creating a virtualenv for! That was the only hiccup I hit. – Zolnoor May 22 '19 at 20:35
  • This youtube video explained it perfectly - https://www.youtube.com/watch?v=1UDEp90S9h8 - download the correct Linux Wheel file for the version of python required, then use wheel from the command line to extract the files, move to a folder named python and then zip it. – yeliabsalohcin Feb 14 '23 at 15:57