24

I have tried to upload my application using servless/lambda function AWS, but i got this issue:

An error occurred: AppLambdaFunction - Unzipped size must be smaller than 262144000 bytes (Service: AWSLambdaInternal; Status Code: 400; Error Code: InvalidParameterValueException; Request ID: 8ea0d887-5743-4db1-96cd-6c5efa57b081).

What is the best way to resolve it?

Look my dependencies:

  "dependencies": {
    "ethereumjs-tx": "^1.3.7",
    "aws-sdk": "^2.4.52",
    "body-parser": "^1.18.3",
    "compression": "^1.7.4",
    "consign": "^0.1.6",
    "cors": "^2.8.5",
    "express": "^4.16.4",
    "helmet": "^3.16.0",
    "moment": "^2.24.0",
    "openzeppelin-solidity": "^2.3.0",
    "serverless": "^1.48.2",
    "serverless-http": "^1.9.1",
    "serverless-offline": "^4.9.4",
    "truffle": "^5.1.9",
    "truffle-hdwallet-provider": "^1.0.17",
    "web3": "^1.2.5-rc.0"
  },

Serverless.yml:

provider:
  name: aws
  runtime: nodejs8.10
  stage: v1
  region: us-east-1
  timeout: 30
  memorySize: 512
  package:
    excludeDevDependencies: true
    exclude:
      - .git/**
      - .vscode/**        
      - venv/**

functions:
  app:  
    handler: handler.run
    events:
      - http:
          path: /
          method: ANY
          cors: true
      - http:
          path: /{proxy+}
          method: ANY
          cors: true

plugins:
  - serverless-offline
John Rotenstein
  • 241,921
  • 22
  • 380
  • 470
Danilo
  • 381
  • 1
  • 2
  • 4
  • i put on "exclude" : exclude: - node_modules/** - .git/** - .vscode/** - venv/** But didnt exclude the node_modules folder... ( and i dont know if it will resolve the issue, because i need to have this dependences on S3) – Danilo Jan 27 '20 at 13:07
  • Any layers in your lambda? They count towards the size limit as well. – 404 Jan 27 '20 at 15:16
  • 5
    `"aws-sdk": "^2.4.52"` unless you need a specific version of aws sdk, do not include this in your dependencies as the aws sdk is already available in the lambda environment without you having to install it. Put it in your dev dependencies instead. – 404 Jan 27 '20 at 15:21
  • 1
    I don't use many libraries so don't recognise most of the ones you have listed, but is "serverless*" to do with the serverless transformation of your template, i.e. required on your side but not in your final product? If so those should be moved to dev dependencies as well. Anything you don't need in your final lambda should be in dev dependencies. – 404 Jan 27 '20 at 15:26
  • As @404 says, move the unneeded dependencies (at least aws-sdk and serverless*) out of dependencies and into devDependencies. – jarmod Jan 28 '20 at 02:13
  • 1
    Guys, thanks so much! I did it, putting some packages that is not necessary on deploy to devDependencies, and now its worked!!! Tkss everybody! – Danilo Feb 05 '20 at 12:19

8 Answers8

41

Use the directive exclude at your serverless.yml file. In case of Python, I've been used it as follows:

package:
  exclude:
    - node_modules/**
    - venv/**

The build process will exclude them from the build before sending to AWS.

Tip I got in this issue at Github. The documentation for this directive is detailed here.

Adriano Laranjeira
  • 532
  • 4
  • 5
2

You can use module bundlers to package the code.

Using module bundlers such as webpack

You can consider using plugins like serverless-webpack. The serverless-webpack plugin is using webpack to build the project and it will only include the bare minimum files required to run your application. It will not include the entire node_modules directory. so that your deployment package will be smaller.

a note about using of Lambda layers

Like others mentioned, you can use the layers and move some of the libraries and code to the layer. Layers are mainly used to share code between functions. The unzipped deployed package including layers cannot exceed 250MB.

hope this helps.

References:

https://github.com/serverless-heaven/serverless-webpack

https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html#configuration-layers-path

Arun Kamalanathan
  • 8,107
  • 4
  • 23
  • 39
1

I've had success resolving this error message using the serverless-esbuild plugin, and configuring it as follows in serverless.yml:


service: service_name
frameworkVersion: '3'
provider:
  name: aws
  runtime: nodejs12.x

plugins:
  - serverless-esbuild

custom:
  esbuild:
    bundle: true
    minify: false
    sourcemap: true
    exclude: 'aws-sdk'
    target: node14
    define:
      'require.resolve': undefined
    platform: node
    concurrency: 10
user3264341
  • 89
  • 1
  • 8
1

Recently i faced the same issue, my total package size was more than 40mbs and it was also including the venv (python virtual environment) folder that resides in the project directory. I excluded it and build size got reduced to 16 mbs. and the project was deployed successfully. I added the following in serverless.yaml

package:
  patterns:
    - '!node_modules/**'
    - '!venv/**'
    - '!apienv/**'
    - '!__pycache__/**'
Saim Abdullah
  • 174
  • 14
1

I ran into this issue once I started depending on the boto3 library.

After trying lots of different things, I learned that this package comes pre-installed with the AWS Python runtime so you can either leave the dependency undeclared or make it a dev dependency.

danvk
  • 15,863
  • 5
  • 72
  • 116
0

See my answer here

You can deploy a Lambda function using a Docker image and it bypasses this problem, allowing a function with its dependencies to be as large as 10 gb.

Wesley Cheek
  • 1,058
  • 12
  • 22
0

Adding exclude under package is deprecated. We can use pattern to remove node_modules.

Example to remove files in the serverless.yml

...remaining props
package:
  patterns:
    - '!.git/**'
    - '!test/**'
    - '!e2e/**'
    - '!src/**'
    - '!node_modules/**'

Deprecation for exclude and Pattern

Yogii
  • 1
  • i tried this but get an error ```Error: ENOTEMPTY: directory not empty, rmdir``` – Dworo Aug 18 '22 at 08:59
  • Can you give more detailed information to debug on the error? Or we can open separate stackoverflow thread to fix your error! – Yogii Aug 19 '22 at 05:30
-1

You can load larger packages into AWS Lambda indirectly using s3:

  • Load your package into a bucket/key on S3
  • In the Lambda console choose Function Code -> Code Entry Type -> Upload a file from S3
debug
  • 1
  • 1
  • 4
    I don't think you can have a package over 250MB, no matter where you put it. Serverless is probably uploading it to S3 first anyway. – gandaliter Jan 27 '20 at 18:59
  • That is the limit, you're right. 250MB is a lot. They must be pulling in a lot of extra stuff. – debug Feb 03 '20 at 19:09
  • 1
    If 250MB is the limit & if I required to use machine learning libs like xgboost, what is the way out? The zipped version of xgboost is about 150 MB and unable create a layer that has only this library – nbs Nov 16 '20 at 06:23
  • @NBhat you should look for another alternative if you can't fit everything in a lambda, some limits are soft (you can request an increase by talking to AWS) while others (like deployment file size) are not https://docs.aws.amazon.com/lambda/latest/dg/gettingstarted-limits.html – jccampos Dec 01 '20 at 21:05
  • 1
    Refer to this [comment](https://stackoverflow.com/a/62235753/4149861); I'm using a combination of both: aws layers (for scipy numpy pandas that's within limit) & extracting xgboost lib into /tmp/pkg/ directory along with adding /tmp/pkg to sys.path. I was successful with this approach. – nbs Dec 04 '20 at 14:18