1

I want to deploy working python project in pycharm to aws lambda. The project is using google-ads library to get some report data from google ads.

I tried deploying lambda by importing complete project as a zip file by zipping all the folders/files inside the project and not the project folder itself. But i got the following error:

{
"errorMessage": "Unable to import module 'main': cannot import name 'cygrpc' from 'grpc._cython' (/var/task/grpc/_cython/__init__.py)",
"errorType": "Runtime.ImportModuleError",
"stackTrace": []
}

Assuming that google-ads library is working and that something is wrong with grpc(btw google-ads includes grpcio and stuff on its own), i tried to create a layer for grpcio, cython, cygrpc but the error remains same.

I create projects/layers in aws lambda and they work. I dont know what i am doing wrong here.

Any help would be really appreciated!

versions: google-ads-14.1.0, python-3.9, grpcio-1.43.0

Abdullah Hafeez
  • 83
  • 1
  • 9

2 Answers2

2

Answering my own question after a lot of workaround. I have made it generic so anyone can use it.

I believe you can fix any type of ImportModuleError as long as your deployment package's file structure, your code and architecture is ok, only then you can deploy and run your code successfully. To fix your structure and architecture, follow steps below:

1- Install "ubuntu 18.04 LTS" from microsoft store (Windows 10).

2- Open CMD and run following commands:

  • ubuntu1804
  • Enter password or create user if asked.
  • cd /mnt/c You can choose any of your drive. I chose C.
  • mkdir my-lambda-folder Create project folder.
  • cd my-lambda-folder Enter into project folder.
  • touch lambda_function.py Create file called lambda_function.py
  • Now copy and paste your code into file you just created i.e lambda_function.py
  • pip install --target ./package your-module-name
  • For Example: pip install --target ./package google-ads will install google-ads module inside folder 'package'. The folder 'package' will be created automatically if not found.
  • cd package
  • zip -r ../my-deployment-package.zip . This will create deployment package with the installed library at the root of your project folder i.e my-lambda-folder.
  • cd .. go back to the root of your project folder.
  • zip -g my-deployment-package.zip lambda_function.py Add your lambda function to the deployment package you just created i.e my-deployment-package.zip.
  • (Optional) In my case i was using google-ads and to run my code i needed google-ads.yaml file too in my deployment package. So i ran additional command zip -g my-deployment-package.zip google-ads-yaml (i already pasted this file in my project folder).

3- Upload my-deployment-package.zip to your lambda function in AWS console and you are good to go.

Abdullah Hafeez
  • 83
  • 1
  • 9
  • would you please explain what you did there and what is the difference between your first approacha and this one? – Amir Apr 28 '22 at 10:48
  • @Amir I successfully installed python modules, built the project and deployed it to AWS lambda. Previously i was uploading same project with modules installed inside venv and also in the root directory ( Example Command: "pip install googleads -t . ) but still got the error, these methods were working fine with other modules but not with google-ads and i don't know why. – Abdullah Hafeez Apr 29 '22 at 13:12
  • I think the issue was arising because cygrpc module was not installing correctly or there was some path malfunctioning. – Abdullah Hafeez Apr 29 '22 at 13:15
  • So basically we could say the problem was solved by installing the modules on Linux. Windows was the problem. – Amir Apr 30 '22 at 14:22
  • 1
    Well, this was a tricky part in the question and i would say it is very strange in this particular case. Because i tried the same thing using my windows machine, a linux based ec2 instance and also with aws cloud9 which is also based on linux, even verified folder structure but none of them worked. One thing which was common in all three cases was that the code was working but as soon as i deploy it to lambda, it gives me the error. Only above method/answer worked which was again based on linux. :D – Abdullah Hafeez May 01 '22 at 03:37
0

For me, it worked just by downloading the packages with pip on ubuntu on docker and packing and uploading them on AWS.

cottontail
  • 10,268
  • 18
  • 50
  • 51
Amir
  • 399
  • 1
  • 7
  • 25