How to deal with large WAR files in AWS?

Question

What I have done ?

Developed a Web app using JSP which allows user to register , login and upload file to AWS S3. I am deploying this app to aws using Elastic Beanstalk by uploading war file of app. For login and register modules I have used RDS and it is working fine.

Problem

When I want to upload files to S3 , I need to use AWS SDK jar and its supporting JAR files in WEB app. When I finished with development part and exported war file , it was around 75 MB. So problem is that if I change anything to app , I need to upload this large size war to aws. Size of the war file is effected due to JAR files of AWS SDK.

What are the options to deal with such situation ?

Answer 1

Use build tools like Maven. This will ensure the dependency files downloaded in the server's local maven repository. So it makes the upload size of your project, reduced. Here is a official quick guide

Answer 2

The steps to deploy partial (thus smaller) war files with Elastic Beanstalk:

1. create a separate package (e.g. dist-lib.zip) w/ the external libraries in use
2. upload this package to S3
3. create the Elastic Beanstalk deployment config files for downloading and extracting the external libraries on the EC2 nodes
4. create the war file w/o the external libraries
5. deploy the war file as usual

1. dist-lib.zip

Excerpt from build.gradle of the web module (the version number is removed from the path within the zip):

apply plugin: 'java-library-distribution'

distTar.enabled = false
distZip.enabled = hasProperty('dist') // 

def subPrjs = rootProject.subprojects.collect { it.name } - project.name

//...

distributions.main {
    contents.exclude subPrjs.collect { "$it*.jar" }
    contents.exclude "tomcat-embed-*"
    contents.exclude "tomcat-annotations-api-*"
    contents.exclude "spring-boot-starter-tomcat-*"
}

distZip {
    baseName += '-lib'
    version = null
    doLast {
        def f0 = archivePath;
        version = props.ver; // replace w/ your version number
        if (archivePath.exists()) {
            archivePath.delete()
        }
        f0.renameTo(archivePath.path)
    }
}

Create the zip file w/: gradle distZip -Pdist=true.

2. Upload dist-lib.zip to S3

From aws-cli: aws s3 cp YOUR_MODULE/build/distributions/YOUR_MODULE-lib-YOUR_VERSION.zip s3://YOUR_BUCKET/dist/dist-lib.zip

I'm suggesting securing the bucket:

1. it won't generate extra external traffic for you;
2. if something goes wrong and you accidentally upload some sensitive data, it won't cause any damage.

3. EB config

The configuration details for accessing a private S3 bucket are described in Storing Private Keys Securely in Amazon S3.

You need the following files to automatically download, extract and add the external libraries to the extracted war package before your application is copied into Tomcat's webapps.

deploy/eb/app-res.config:

Resources:
  # Use instance profile to authenticate to S3 bucket that contains the private key
  AWSEBAutoScalingGroup:
    Metadata:
      AWS::CloudFormation::Authentication:
        S3Auth:
          type: "s3"
          buckets: ["elasticbeanstalk-us-east-1-169305339676"]
          roleName:
            "Fn::GetOptionSetting":
              Namespace: "aws:autoscaling:launchconfiguration"
              OptionName: "IamInstanceProfile"
              DefaultValue: "aws-elasticbeanstalk-ec2-role-dev"

files:
  # distribution libs
  /tmp/dist-lib.zip:
    mode: "000644"
    owner: tomcat
    group: tomcat
    authentication: "S3Auth"
    source: https://s3.amazonaws.com/YOUR_BUCKET/dist/dist-lib.zip

deploy/eb/dist-lib.config:

files:
  /opt/elasticbeanstalk/hooks/appdeploy/pre/10_lib_extr.sh:
    mode: "000755"
    owner: root
    group: root
    content: |
      #!/usr/bin/env bash
      rm -rf /tmp/dist-lib
      unzip  /tmp/dist-lib.zip -d /tmp/
      mv     /tmp/dist-lib/lib /tmp/deployment/application/ROOT/WEB-INF/lib 

4. war package

Another part from build.gradle of the web module (the submodule (subproject) classes are included directly in WEB-INF/classes):

apply plugin: 'war'

jar.enabled = false

def env = project.hasProperty('env') ? project.getProperty('env') : 'lcl' // profile: set w/ "-Penv=..."
def ebDirs = ["$rootDir/deploy/eb-$env", "$rootDir/deploy/eb", "$rootDir/deploy/eb/_nginx"] // env. specific config first

// ...

war {
    duplicatesStrategy = 'fail'
    //rootSpec.exclude subPrjs.collect { "**/$it*.jar" } // exclude subproject jars only
    rootSpec.exclude "**/*.jar" // exclude dependencies => they must be downloaded and extracted during deployment
    from(subPrjs.collect { project(":$it").sourceSets.main.output }) {
        duplicatesStrategy = 'exclude' // in case of one output dir for multiple sourceSets
        into "WEB-INF/classes"
    }
    from(ebDirs) {
        duplicatesStrategy = 'exclude' // in case of env. spec. config
        exclude "**/_*"
        into ".ebextensions"
    }
}

5. Deploy

The above solution was tested w/ gradle v4.10.2 and aws-cli v1.16.44 (both w/ eb create and eb deploy).