22

I have based a small lambda function off the cdk workshop here. I'm writing the lambda function in typescript, deploying via a pipeline which creates a cloud formation stack containing the lambda function.

I'm trying to use the sdk v3 in lambda, as demoed here. But then I see conflicting documentation here.

Are these errors because I'm trying to use V3 and I shouldn't, or for some other reason? The handler is set correctly, the function runs but fails with the error:

{
    "errorType": "Runtime.ImportModuleError",
    "errorMessage": "Error: Cannot find module '@aws-sdk/client-sns'\nRequire stack:\n- /var/task/ReceiveMessageLoraThing.js\n- /var/runtime/UserFunction.js\n- /var/runtime/index.js",
    "stack": [
        "Runtime.ImportModuleError: Error: Cannot find module '@aws-sdk/client-sns'",
        "Require stack:",
        "- /var/task/ReceiveMessageLoraThing.js",
        "- /var/runtime/UserFunction.js",
        "- /var/runtime/index.js",
        "    at _loadUserApp (/var/runtime/UserFunction.js:100:13)",
        "    at Object.module.exports.load (/var/runtime/UserFunction.js:140:17)",
        "    at Object.<anonymous> (/var/runtime/index.js:43:30)",
        "    at Module._compile (internal/modules/cjs/loader.js:999:30)",
        "    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10)",
        "    at Module.load (internal/modules/cjs/loader.js:863:32)",
        "    at Function.Module._load (internal/modules/cjs/loader.js:708:14)",
        "    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:60:12)",
        "    at internal/main/run_main_module.js:17:47"
    ]
}

The file is deployed as js, with the correct handler set. If I comment out the require statement, it works fine:

// works
"use strict";
//const sns = require("@aws-sdk/client-sns");
exports.handler = async (event) => {
    console.log("hello");
    return true;
}

// doesn't work
"use strict";
const sns = require("@aws-sdk/client-sns");
exports.handler = async (event) => {
    console.log("hello");
    return true;
}

There are no node_modules or layers generated by using the code from this workshop, but before I go there I want to know if I can actually use V3 on lambda yet.

A-Sharabiani
  • 17,750
  • 17
  • 113
  • 128
thinkOfaNumber
  • 2,581
  • 3
  • 27
  • 46
  • 1
    FWIW you can inspect what's in an AWS Lambda runtime by doing `docker run --rm -it --entrypoint /bin/bash amazon/aws-lambda-nodejs[:version]`. The pre-installed packages available in the Node 14 version are in `/var/runtime/node_modules`, which contains only `aws-sdk`. – Andy Dec 01 '21 at 02:18
  • There are also packages `npm` depends on in `/var/lang/lib/node_modules/npm/node_modules`, but you can't require them by name from your handler, and it would probably be a bad idea to rely on them – Andy Dec 01 '21 at 02:20

1 Answers1

29

Yes, you can use AWS SDK v3, just like any other JS library.

The Node environment for Lambda comes with installed AWS SDK v2 (the previous one), as you can see here: https://docs.aws.amazon.com/lambda/latest/dg/lambda-nodejs.html

So to use v3, you should bundle it with your application as with any other dependency, and deploy the bundle (https://docs.aws.amazon.com/lambda/latest/dg/nodejs-package.html).

Actually, even with SDK v2 being available on the Lambda env, it was still a best practice to bundle it by yourself with your app and deploy it. This way you would always use the version you specified, and not the one that is on Lambda and can be updated any time. Even if they do not make any incompatible changes, it's always possible that the new SDK version, not tested with your specific app, will have some bug that will break your Lambda (very unlikely, but possible).

Edit (2022-11-22): Node.js 18 Lambda runtime includes SDK v3 and does not include v2 (https://aws.amazon.com/blogs/compute/node-js-18-x-runtime-now-available-in-aws-lambda/).

Radzikowski
  • 2,377
  • 4
  • 27
  • 39
  • It will make my project a bit more complicated, but good advice, thanks. Any advice on building multiple separate lambda functions in the same project? Do I have to duplicate/install node_modules for each one? (I'm using the CDK to deploy) – thinkOfaNumber Feb 15 '21 at 01:55
  • 1
    I did not use CDK (only Serverless Framework) so I don't know how this exactly works, but I don't see a reason to multiply node_modules. It should be possible to create multiple functions in single project. – Radzikowski Feb 15 '21 at 06:58
  • While you can use v3 in lambda, you have to include it yourself, I don't think it's ready for lambda usage yet. I have been trying to download an S3 object with little success. I can get it to work in NodeJS but not when I deploy it to a lambda. – Hayden Feb 19 '21 at 19:18
  • Consider using Layers to provide the required v3 sdk modules. That won't affect bundling for you Function. – bdargan Feb 19 '21 at 21:38
  • @bdargan Layers are great, but stuffing all the aws sdk clients you need into a layer effectively makes tree shaking worthless, and increases your lambda cold start times. – JHH Apr 03 '21 at 20:55
  • 1
    @JHH agree and as they want to use sdk v3 they are going to include them one way or another. I suggest they provide them to their lambda in an isolated layer and not bundle together with rest of module deps. In a separate layer they may share with other lambdas and when v3 is provided out of the box OP can either drop the layer or continue maintain specific v3 version themselves. – bdargan Apr 04 '21 at 02:04
  • 3
    Is there any reason they do not include the aws-sdk v3 as a acpkage the same way they do it with v2? – zirkelc Jul 06 '21 at 15:58
  • 3
    The downside of always deploying a specific SDK version via layer, or otherwise, is that you lose the benefit of the AWS Lambda service auto updating the SDK dependency for you. That's a potential no-cost security benefit whose value needs to be weighed against the risk of a breaking change in the SDK. – jarmod Oct 22 '21 at 15:20
  • 2
    @zrkl, AWS seem to have gone to a lot of trouble in v3 to separate out all their packages. They've always advised people to bundle up a specific SDK version rather than relying on the lambda runtime version. I'd imagine they might be intentionally not exposing v3 so that people follow this advice; afterall I'm sure it would be fairly trivial for them to just bake v3 into the lambda runtime e.g. with a Docker build. As for v2, they probably have to leave that in there for backwards compatibility. I may be wrong, maybe v3 will be part of the runtime in future, but I'm not holding my breath. – Colm Bhandal Jul 20 '22 at 16:25
  • 1
    I'd like to advice against the "general rule" of bundling the SDK with your lambda. Given that it makes your lambda about 100M larger, there many reasonable use cases where you definitely do not want to bundle it with your application. The risk of something going wrong with minor updates is very slim, but the upside of not bloating your lambda code is HUGE. – enanone Sep 05 '22 at 09:48
  • 1
    my project without @aws-sdk, zip = 168KB ... adding *only* @aws-sdk/client-secrets-manager, zip = 2,152 KB ... intolerable ... build it in, build several versions in if need be... – Reinsbrain Oct 26 '22 at 20:50