2

I'm facing issue in uploading the maven deployment package to Amazon s3.

From Eclipse, I'm able to generate the .jar file successfully, however I'm facing issue in uploading to server.

Here is my Java code:

package main.java.mavantestproj;

import java.util.Map;

import com.amazonaws.auth.profile.ProfileCredentialsProvider;
import com.amazonaws.regions.Regions;
import com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient;
import com.amazonaws.services.lambda.runtime.Context;

public class LambdaFunctionHandler {


    public String handleRequest(Map<String,Object> input, Context context) {
        context.getLogger().log("Input: " + input);

        AmazonDynamoDBClient client = new AmazonDynamoDBClient(new ProfileCredentialsProvider("mytest"));

        client.setRegion(com.amazonaws.regions.Region.getRegion(Regions.US_WEST_2));

        client.describeTable("ProductCatalog");
        // TODO: implement your handler
        return null;
    }

}

in target folder i have got 2 jar's. ie lambda-java-example-1.0-SNAPSHOT.jar & original-lambda-java-example-1.0-SNAPSHOT.jar

In this first jar is 35MB & second one is 4KB. I'm not getting which one to upload to S3 to run my lambda function.

TylerH
  • 20,799
  • 66
  • 75
  • 101
Naruto
  • 9,476
  • 37
  • 118
  • 201
  • Can you post what your POM dependencies look like? 35 MB seems a bit large if all you are interacting with is DynamoDB. You should be able to get that down to just 11 MB based on my experience. – JaredHatfield Mar 20 '16 at 18:01

3 Answers3

5

You definitely need the large "uber-jar" so your dependency classes will be included, but there is an alternative way to package things up for AWS-Lambda using the Maven assembly plugin instead of the Shade plugin. You end up with an AWS lambda deployment package in .zip format instead of a single .jar file. It will look a little more like a JEE .war file with all the original .jar dependencies kept intact, and you can include other stuff like properties files that end up unpacked in the file-system where the lambda runs (which may be a little easier to find and load in your code). If you're interested in the details, there's a blog post about it here: http://whirlysworld.blogspot.com/2016/03/aws-lambda-java-deployment-maven-build.html Also, packaging a Lambda function app this way makes it WAY easier to peek into the zip file and figure out which dependency jars are being included, and then identify the ones you might be able to exclude.

This still doesn't get Maven to handle the actual deployment of the package to AWS (create or update). Deploying, and capturing the deployment info (ARN, API-gateway app-id-url, etc.), seems to be the next thing for which Amazon hasn't provided a very clear answer or solution.

Community
  • 1
  • 1
Hubschrauber
  • 91
  • 2
  • The related Maven Assembly Plugin [documentation on `dependencySet`](http://maven.apache.org/plugins/maven-assembly-plugin/assembly.html#class_dependencySet) was supplemental in my case (eg. `excludes` directive). – Tommy Stanton Aug 11 '16 at 21:20
  • If you want to figure out why particular dependencies are being included the command `mvn dependency:tree` is very useful. – tgdavies Oct 14 '17 at 00:13
0

The larger JAR file that is being generated includes all of the library dependencies baked in. This is the one you will want to upload to S3 for use by AWS Lambda as these dependencies are required to run.

If you want to make this file smaller you can ensure you are only including libraries you need and remove any unnecessary ones. A common way to do this is with the AWS SDK only include the libraries for the specific services you need to call such as DynamoDB instead of including the entire AWS SDK.

JaredHatfield
  • 6,381
  • 2
  • 29
  • 32
  • Hi Thats great, thank you for reply. I have one doubt. If i directly call amazon s3 bucket from eclipse directly, everytime my code takes 25 mins to upload being 6mbps broadband speed. If i use maven & if i want to include amazon aws related libs for the project only, this will generate 34MB of jar file, this does takes again 15 mins to upload. Everytime if i make any small code change also, i need to follow the same step for both options. This is very tedious right?. or any alternatives you will suggest please? – Naruto Mar 20 '16 at 17:54
  • A 35 MB upload at 6 Mbps should only take 47 seconds. The obvious solution is to not use your internet connection then. The way I deploy is using Jenkins running on an EC2 instance so all I need to transfer locally is the code to my git repository and Jenkins takes care of the build and uploading to S3 and making the necessary API calls to deploy the code. – JaredHatfield Mar 20 '16 at 17:58
  • ok, fine, but if i deploy code via eclipse to s3 bucket. My eclipse also takes longer time. Even a small java file change also, upload the complete binary to s3 bucket. So according to you alternative is Jenkin only?. i'm very new to this, please correct me if i m wrong. Thanks – Naruto Mar 20 '16 at 18:02
  • You have to deploy the larger version since that is what is required to actually run the code in Lambda. However, I'm guessing you are including the "aws-java-sdk" dependency when all you may need is "aws-java-sdk-dynamodb" which will reduce the deployment size. – JaredHatfield Mar 20 '16 at 18:05
  • Yes, you are right, i have removed that and i used this one ie http://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk-dynamodb.. this one reduced the size from 35mb to 5 MB, is there any other LIB which is related to dynamodb functions in maven which could further reduce my jar file size? – Naruto Mar 20 '16 at 18:17
0

It seems standalone jar file built using shade plugin is sufficient as per this AWS documentation

sandeepkunkunuru
  • 6,150
  • 5
  • 33
  • 37