When attempting to run sharp inside an AWS Lambda function, I keep getting the following error:
darwin-x64' binaries cannot be used on the 'linux-x64' platform. Please remove the 'node_modules/sharp/vendor' directory and run 'npm install'
I deploy my serverless applications with Serverless Framework from my MacBook Pro. How do I fix this problem?
Kudos to stdunbar for steering me in the right direction.
When installing sharp on MacOS via NPM the normal way (i.e.: npm i sharp --save), the installer automatically adds binaries for OS X. But AWS lambda functions run on Linux 2 machines with x64 processors and this is why we get this error.
To fix you must first uninstall sharp completely and then run:
npm install --arch=x64 --platform=linux sharp
Note:
version 0.25 no longer works with the target flag. This used to work:
npm install --arch=x64 --platform=linux --target=10.15.0 sharp
Then deploy as usual from Serverless Framework with sls deploy
Side Note:
Sharp is EXTREMELY FAST!!! Before using sharp, I was using another image resizing utility named Jimp. It did the job, but was quite slow. To prevent timeout errors, I had to increase the memory size from 128 to 512 and the timeout from 5 seconds to 30 seconds just to handle a typical 1 megabyte image.
Here is a comparison between the two for resizing a 1.2Mb picture down to 600x400 using the same configuration:
Jimp -> used 512Mb of memory and AWS billed me for 14300 ms.
Sharp -> used 132 MB of memory and AWS billed me for 800 ms.
That's more than 14x faster than Jimp!!!
If you are using Docker like I am and come across this issue.
Most likely your node_modules folder is being copied from your host machine into the docker container.
Why? Because your node_modules folder downloads modules specific to the architecture of your machine (in my case Mac), when you take these packages and try to run them in the container OS, it causes issues because the binaries are for mac (darwin-x64) and it tries to run them on linux (linux-x64) inside the container
To fix it, create a file named .dockerignore in the same directory as your Dockerfile and update it accordingly.
.dockerignore
node_modules/
Dockerfile
FROM node:latest
WORKDIR /usr/src/app
COPY package*.json ./
RUN ["npm", "install"]
COPY . .
CMD ["npm", "start"]
It will now ignore the node_modules folder on the host machine, and re-populate the modules once command npm install inside the Dockerfile has run.
I found an easier solution when building from MacOS or Windows, I use a Makefile with SAM and provide linux as a target for the NPM install.
template.yaml
MyFunction:
Type: AWS::Serverless::Function
Properties:
CodeUri: mycode/
Handler: handler.handler
Runtime: nodejs12.x
MemorySize: 512
Timeout: 5
Role: !GetAtt EdgeLambdaRole.Arn
Metadata:
BuildMethod: makefile
mycode/makefile
build-MyFunction:
cp *.js $(ARTIFACTS_DIR)
cp package.json $(ARTIFACTS_DIR)
cp package-lock.json $(ARTIFACTS_DIR)
cd $(ARTIFACTS_DIR) && npm install --production --arch=x64 --platform=linux
Make sure you use TABS in your makefile
If you like to yarn, set via environment variables
NOTE: I clean up the existing platform/arch first.
# For macOS/Windows, we can install natively
yarn remove sharp
yarn add sharp
# For AWS Lambda, we need to specify this when we package:
yarn remove sharp
npm_config_platform=linux npm_config_arch=x64 yarn add sharp
I was facing the same issue, when i do npm install --arch=x64 --platform=linux sharp in CMD in windows machine.
Fixed it by opening the CMD in Administrator mode, i did not find any error in running npm install --arch=x64 --platform=linux sharp
If you use serverless along with serverless-webpack, you must use this custom-config in your serverless.yml file:
custom:
# ...
webpack:
# ...
packagerOptions:
scripts:
- npm install --arch=x64 --platform=linux sharp
If you use SAM you could build your function inside an AWS Lambda-like Linux Docker container:
sam build -u
For anyone stumbling upon this post now. I've accomplished this by copying my package.json file into an AWS Cloud9 IDE and simply running npm install. From there, just download the node_modules/ folder.
