60

is there ANY way to execute aws-cli inside AWS Lambda? It doesn't seem to be pre-installed. (I've checked with "which aws" via Node.js child-process, and it didn't exist.)

gvasquez
  • 1,919
  • 5
  • 27
  • 41
Hirofumi Okino
  • 975
  • 1
  • 10
  • 22
  • 3
    Any progress on this? I'm looking for the same question, I need to use command "aws opsworks register". It sucks that they don't provide this ability in all of Lambda languages – Casper Feb 26 '16 at 22:59
  • Why would you want to use the CLI? What is it that you're trying to do that cannot be achived using the AWS SDK? – st.huber Jan 10 '22 at 11:30

8 Answers8

23

Now we can use Layers inside Lambda. Bash layer with aws-cli is available at https://github.com/gkrizek/bash-lambda-layer

handler () {
    set -e

    # Event Data is sent as the first parameter
    EVENT_DATA=$1

    # This is the Event Data
    echo $EVENT_DATA

    # Example of command usage
    EVENT_JSON=$(echo $EVENT_DATA | jq .)

    # Example of AWS command that's output will show up in CloudWatch Logs
    aws s3 ls

    # This is the return value because it's being sent to stderr (>&2)
    echo "{\"success\": true}" >&2
}
Jai
  • 694
  • 5
  • 15
14

Not unless you include it (and all of its dependencies) as part of your deployment package. Even then you would have to call it from within python since Lambda doesn't allow you to execute shell commands. Even if you get there, I would not recommend trying to do a sync in a Lambda function since you're limited to a maximum of 5 minutes of execution time. On top of that, the additional spin-up time just isn't worth it in many cases since you're paying for every 100ms chunk.

So you can, but you probably shouldn't.

EDIT: Lambda does allow you to execute shell commands

Jordon Phillips
  • 14,963
  • 4
  • 35
  • 42
  • 5
    See https://alestic.com/2016/11/aws-lambda-awscli/ for a good overview on how to accomplish bundling the aws-cli python package for use in lambda – Noah Dec 27 '16 at 19:20
  • Actually using aws cli **IS** a good idea when using Lambda for a Cloudformation Custom resource for example – Yves M. Oct 24 '18 at 09:55
10

aws-cli is a python package. To make it available on a AWS Lambda function you need to pack it with your function zip file.

1) Start an EC2 instance with 64-bit Amazon Linux;

2) Create a python virtualenv:

mkdir ~/awscli_virtualenv
virtualenv ~/awscli_virtualenv

3) Activate virtualenv:

cd ~/awscli_virtualenv/bin
source activate

4) Install aws-cli and pyyaml:

pip install awscli
python -m easy_install pyyaml

5) Change the first line of the aws python script:

sed -i '1 s/^.*$/\#\!\/usr\/bin\/python/' aws

6) Deactivate virtualenv:

deactivate

7) Make a dir with all the files you need to run aws-cli on lambda:

cd ~
mkdir awscli_lambda
cd awscli_lambda
cp ~/awscli_virtualenv/bin/aws .
cp -r ~/awscli_virtualenv/lib/python2.7/dist-packages .
cp -r ~/awscli_virtualenv/lib64/python2.7/dist-packages .

8) Create a function (python or nodejs) that will call aws-cli:

For example (nodejs):

var Q = require('q');
var path = require('path');
var spawn = require('child-process-promise').spawn;    

exports.handler = function(event, context) {

    var folderpath = '/folder/to/sync';
    var s3uel = 's3://name-of-your-bucket/path/to/folder';

    var libpath = path.join(__dirname, 'lib');
    var env = Object.create(process.env);
    env.LD_LIBRARY_PATH = libpath;

    var command = path.join(__dirname, 'aws');
    var params = ['s3', 'sync', '.', s3url];
    var options = { cwd: folderpath };

    var spawnp = spawn(command, params, options);

    spawnp.childProcess.stdout.on('data', function (data) {
        console.log('[spawn] stdout: ', data.toString());
    });

    spawnp.childProcess.stderr.on('data', function (data) {
        console.log('[spawn] stderr: ', data.toString());
    });

    return spawnp
    .then(function(result) {

        if (result['code'] != 0) throw new Error(["aws s3 sync exited with code", result['code']].join(''));

        return result;

    });

}

Create the index.js file (with the code above or your code) on ~/awscli_lambda/index.js

9) Zip everything (aws-cli files and dependencies and your function):

cd ~
zip -r awscli_lambda.zip awscli_lambda
6

Now you can simply run it as Docker container within lambda along with AWS CLI.

irscomp
  • 2,900
  • 1
  • 16
  • 12
  • To add some context - AWS Lambda now supports container images - https://aws.amazon.com/blogs/aws/new-for-aws-lambda-container-image-support/ – Sashi Sep 20 '21 at 21:29
4

You can use the AWS node.js SDK which should be available in Lambda without installing it.

var AWS = require('aws-sdk');
var lambda = new AWS.Lambda();
lambda.invoke({
    FunctionName: 'arn:aws:lambda:us-west-2:xxxx:function:FN_NAME',
    Payload: {}, 
  },
  function(err, result) {
    ...
});

As far as I can tell you get most, if not all the cli functionality. See the full documentation here: http://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/Lambda.html

Ryan
  • 5,845
  • 32
  • 27
  • 4
    Thanks Ryan, but I want some functions which do NOT exit in node.js SDK and do exist in aws-cli, because it seems easier to use aws-cli in some occasions, such as s3 sync across buckets. – Hirofumi Okino Nov 04 '15 at 06:23
  • Ah, I see now. In that case it seems you would need to install it on every run which sounds kind of painful and inefficient. – Ryan Nov 04 '15 at 15:09
  • you cant install it, yum and pip install is not available in nodejs environment – halil Aug 26 '16 at 19:47
  • The question is about aws cli, not aws sdk – Yves M. Oct 24 '18 at 09:56
2

you can try this. I got this working for me.

1- add the AWS CLI layer

https://harishkm.in/2020/06/16/run-aws-cli-in-a-lambda-function/

2- add a lambda and run the following commands to run any AWS CLI command line.

https://harishkm.in/2020/06/16/run-bash-scripts-in-aws-lambda-functions/

function handler () {
    EVENT_DATA=$1
    
    DATA=`/opt/awscli/aws s3 ls `
    RESPONSE="{\"statusCode\": 200, \"body\": \"$DATA\"}"
    echo $RESPONSE
}
Anh Nhat Tran
  • 561
  • 1
  • 6
  • 18
Tahir Aziz
  • 31
  • 1
1

If you are provisioning your Lambda using code, then this is the most easiest way

lambda_function.add_layers(AwsCliLayer(scope, "AwsCliLayer"))

Ref: https://pypi.org/project/aws-cdk.lambda-layer-awscli/

viru
  • 2,081
  • 19
  • 23
-2

I think that you should separate your trigger logic from the action. Put a container with aws cli on another ec2 And use aws lambda to trigger that into an action.

Yoni
  • 106
  • 1
  • 3
  • 2
    This means that you need to have an ec2 instance sitting around at all times rather than implementing the usefulness of lambdas - small scope & run only when needed. – domdambrogia Dec 18 '18 at 18:40