0

I've been coming up a brick wall with getting AWS Lambda to run my code. I'm following https://robertorocha.info/setting-up-a-selenium-web-scraper-on-aws-lambda-with-python/. Trying to get a web scraper working on AWS Lambda that needs selenium and headless chrome. I will say I'm using windows rather than linux/mac.

The tutorial uses lambci to replicate the environment of AWS Lambda and to install linux dependancies. When making the final zip file for uploading to S3 for use with lambda.

When I put the python script into the folder /src as in the tutorial says to do, it wont import the module lambda_function. When I put the python file by itself in the zip it seems to run but this error message comes up, I have changed the environment variable in lambda so that will see lambda_function.py.

{
  "errorMessage": "Unable to import module 'lambda_function': cannot import name 'WinDLL' from 'ctypes' (/var/lang/lib/python3.7/ctypes/__init__.py)",
  "errorType": "Runtime.ImportModuleError"
}

Here are my requirements:

boto3==1.6.18
botocore==1.9.18
selenium==3.141.0
chromedriver-install==1.0.3
numpy==1.18.2
requests==2.23.0

Folder structure of the zip file to upload to S3.

lambda_function.py
/lib
/bin

When I run make docker-run, the code runs fine with no error messages. For some reason I was getting an error message when running the makefile code that builds the lambda package. I did this part manually essentially moving folders and zipping the libraries, python file up. I'm wondering if that is where my error is. I've read somewhere that numpy doesnt integrate well with AWS Lambda, but it uses the linux .whl file when building the image and its not mentioned in the error message.

I've tried googling the error message and there's not much about it to be honest! I'm not actually sure what it means.

Any help would be appreciated. If you need me to post my python code that's fine, but as I said, make docker-run worked fine without error so I'm not sure it's the code per se that is the problem.

Appreciate any help even if it's just to get a handle on what the error message might be about.

John Rotenstein
  • 241,921
  • 22
  • 380
  • 470
AaronS
  • 2,245
  • 2
  • 6
  • 16
  • The best practice for Lambda development is to bundle all dependencies used by your Lambda function, including the AWS SDK. By doing this, your code uses the bundled version and is not affected when the version in the execution environment is upgraded. This is preferable to using the included version of the SDK, since this version can change, and in rare cases might affect compatibility with your code. Read this: https://aws.amazon.com/blogs/compute/upcoming-changes-to-the-python-sdk-in-aws-lambda/ – Jeremy Thompson Mar 28 '20 at 21:35
  • Can you write a Print statement to check if the DLL exists? `os.path.exists(dll_path):`? Also bear in mind WinDLL may have dependencies itself: https://stackoverflow.com/a/7586821/495455 – Jeremy Thompson Mar 28 '20 at 21:49
  • When I write a print statement I get false for WinDLL. I'm not exactly sure what the actual error message means. I know very little about ctypes and WinDLL, seems to be having problem running the python script. – AaronS Mar 28 '20 at 23:03

0 Answers0