Import Error from cyptography.hazmat.bindings._constant_time import lib

Question

So I'm trying to create an aws lambda function, to log in to an instance and do some stuff. And the script works fine outside of lambda, but when I package it using the same instructions as this https://aws.amazon.com/blogs/compute/scheduling-ssh-jobs-using-aws-lambda/ it doesn't work. It throws this error.

libffi-72499c49.so.6.0.4: cannot open shared object file: No such file or directory: ImportError
Traceback (most recent call last):
  File "/var/task/lambda_function.py", line 12, in lambda_handler
    key = paramiko.RSAKey.from_private_key(key)
  File "/var/task/paramiko/pkey.py", line 217, in from_private_key
    key = cls(file_obj=file_obj, password=password)
  File "/var/task/paramiko/rsakey.py", line 42, in __init__
    self._from_private_key(file_obj, password)
  File "/var/task/paramiko/rsakey.py", line 168, in _from_private_key
    self._decode_key(data)
  File "/var/task/paramiko/rsakey.py", line 173, in _decode_key
    data, password=None, backend=default_backend()
  File "/var/task/cryptography/hazmat/backends/__init__.py", line 35, in default_backend
    _default_backend = MultiBackend(_available_backends())
  File "/var/task/cryptography/hazmat/backends/__init__.py", line 22, in _available_backends
    "cryptography.backends"
  File "/var/task/pkg_resources/__init__.py", line 2236, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/var/task/cryptography/hazmat/backends/openssl/__init__.py", line 7, in <module>
    from cryptography.hazmat.backends.openssl.backend import backend
  File "/var/task/cryptography/hazmat/backends/openssl/backend.py", line 15, in <module>
    from cryptography import utils, x509
  File "/var/task/cryptography/x509/__init__.py", line 7, in <module>
    from cryptography.x509.base import (
  File "/var/task/cryptography/x509/base.py", line 15, in <module>
    from cryptography.x509.extensions import Extension, ExtensionType
  File "/var/task/cryptography/x509/extensions.py", line 19, in <module>
    from cryptography.hazmat.primitives import constant_time, serialization
  File "/var/task/cryptography/hazmat/primitives/constant_time.py", line 9, in <module>
    from cryptography.hazmat.bindings._constant_time import lib
ImportError: libffi-72499c49.so.6.0.4: cannot open shared object file: No such file or directory

@intotecho how would I do that? When I install it I install it from pip. The only two commands I do are `pip install pycrypto` and `pip install paramiko` and it works when I run it on my own server. The problem only occurs when I throw it on aws lambda — Luis F Hernandez, Aug 16 '16 at 00:48
`pip freeze` lists all libraries on your server environment. I don't know how to check on AWS. You could ask on the blog where those instructions were posted. — intotecho, Aug 16 '16 at 01:30

Gabriel Totusek · Accepted Answer · 2016-08-17T19:55:09.363

27

The zip commands in that tutorial are missing a parameter. I ran into this exact problem today with pysftp, which is built on paramiko. libffi-72499c49.so.6.0.4 is in a hidden dot directory inside lib64/python2.7/site-packages/.libs_cffi_backend. Depending on how you zipped up the dependencies in your virtualenv, you may have inadvertantly excluded this directory.

First, make sure libffi-devel and openssl-devel are installed on your Amazon Linux instance, otherwise the cryptography module may not be compiling correctly.
```
sudo yum install libffi-devel openssl-devel
```

If those packages were not installed before, delete and rebuild your virtualenv.

Make sure that when you are zipping up your site-packages that you use '.' instead of '*', otherwise you will not be including files and directories that are hidden because their names begin with a period.
```
cd path/to/my/helloworld-env/lib/python2.7/site-packages
zip -r9 path/to/zip/worker_function.zip .
cd path/to/my/helloworld-env/lib64/python2.7/site-packages
zip -r9 path/to/zip/worker_function.zip .
```

edited Aug 17 '16 at 19:55

answered Aug 16 '16 at 04:03

Gabriel Totusek

386
3
5

4

Thank you for this answer! For those who are starting from scratch. The Amazon Linux instance is absolutely an important piece. And you might want to install `python-devel` and `gcc` before the `pip install pycrypto` – warhod Aug 23 '16 at 22:18
1

@warhod yeah! agreed, my issue was completely fixed when I used an amazon linux instance to create the zip file. Lambda runs on Amazon Linux, so make your zip and libraries on Amazon Linux. It just makes sense. – Luis F Hernandez Aug 26 '16 at 14:33
@warhod you better make that `sudo yum install python27-devel` because Lambda uses Python2.7 and if you already have python27-devel installed, requesting python-devel will install python26-devel. Along with that you would get python26 and python26-libs, and then you will be in a world of pain. If you **do** make that mistake, you can use `python27 -m virtualenv projname` and `python27 -m pip install pkgname` – Bruno Bronosky Mar 17 '17 at 02:37
1

Thank you for this! I created a [simple Makefile](https://gist.github.com/RichardBronosky/e682c46bb5c63c3eae04b7a78870c77b) for preparing a ready to upload zip from a virtualenv. I originally had a `*` instead of a `.` and it worked for everything but cryptography. #sigh – Bruno Bronosky Mar 17 '17 at 03:01
1

**NOTE** in modern Amazon Linux, the `lib64` dir is a symlink to `lib`, so you don't need to add both to your zip. – Bruno Bronosky Mar 17 '17 at 16:04
1

Thanks for this tip! I was going crazy trying to get my build scripts working and finally got them to work by copying with `.` instead of `*`! You rock! – Adam Link Aug 25 '17 at 21:44
1

The file which gets missed if you zip with `*` intead of `.` as mentioned by Gabriel in #2 above is `.libs_cffi_backend` – gene_wood Feb 02 '18 at 23:01
This https://askubuntu.com/questions/259383/how-can-i-get-mv-or-the-wildcard-to-move-hidden-files was also a helpful read for me – HeyWatchThis May 27 '19 at 23:00
I wish this got integrated into the original AWS post – Shadi Aug 09 '19 at 11:26

MarSoft · Answer 2 · 2018-04-08T14:11:48.037

My 2 cents: if you want to build&test your lambda function in the environment as similar to actual lambda as possible but still under your control, I would suggest using LambCI's Docker images. They are based on dumps of original lambda filesystem. Also they have build-specific variants (tags build-python2.7 and build-python3.6 are most interesting for us). These images are not very small - more than 500mb - but they allow you to avoid any headache when building.

Important benefit over Amazon Linux is that all package versions etc are the same as on the real lambda.

Here is how I did building myself:

cd PROJECT_DIR
docker run --rm -it -v "$PWD":/var/task lambci/lambda:build-python2.7 bash
### now in docker
mkdir deps
pip install -t deps -r requirements.txt
# now all dependencies for our package are installed to deps/ directory,
# without any garbage like wheel or setuptools - unlike when using virtualenv
zip -r archive.zip MYCODE.py MYMODULE MYMODULE2.py
cd deps
# it's important to use . here, not * - or else some dot-starting directories will be omitted
zip -r ../archive.zip .
exit
### now locally
# just upload archive to lambda, with or without s3

For automating it with GitLab CI, just instruct it to use that same docker image and put these commands in deploy script section:

deploy:
    stage: deploy
    image: lambci/lambda:build-python2.7
    script:
        - mkdir deps
        - pip install -t deps -r requirements.txt
        - zip -r archive.zip MYCODE.py MYMODULE MYMODULE2.py
        - cd deps && zip -r ../archive.zip . && cd ..
        - aws s3 cp archive.zip ${bucket}/${key}
        - aws lambda update-function-code --function-name ${func} --s3-bucket ${bucket} --s3-key ${key}
    variables:
        bucket: ...
        key: ...
        func: ...

score 0 · Answer 3 · answered Mar 05 '21 at 19:14

Another solution that has been working well for me is to install the linux version of the underlying packages that paramiko needs (bcrypt, cffi, cryptography, and PyNaCl) and unzip them yourself.

You can do this with:

pip download PyNaCl --platform manylinux1_x86_64 --no-deps -d lambda_project/python/lib/python3.8/site-packages
pip download cffi --platform manylinux1_x86_64 --no-deps -d lambda_project/python/lib/python3.8/site-packages
pip download bcrypt --platform manylinux1_x86_64 --no-deps -d lambda_project/python/lib/python3.8/site-packages
pip download cryptography --platform manylinux2014_x86_64 --no-deps -d lambda_project/python/lib/python3.8/site-packages

cd lambda_project/python/lib/python3.8/site-packages
unzip \*.whl
rm *.whl

paramiko itself isnt platform dependent so you can do the normal:

pip install paramiko -t lambda_project/python/lib/python3.8/site-packages/.

I put these into a lambda layer so that once I have it working, I can just attach the same one to any of my lambdas that need it but the same concept should work directly on a lambda.

score 0 · Answer 4 · answered Apr 05 '22 at 19:16

0

In my case, I'd accidentally specified in my Lambda deployment that I wanted Python 3.7 when I'd developed and packaged on a machine running Python 3.6. They need to match.

answered Apr 05 '22 at 19:16

MikeS

51
4

Import Error from cyptography.hazmat.bindings._constant_time import lib

4 Answers4

Linked