AWS Lambda Jar unable to zip after adding selenium dependencies in pom

Question

This is a weird error. After adding the selenium dependencies to the pom of my maven project and upload it to a lambda, it says it is unable to unzip the file. However after removing the dependencies, the lambda is able to unzip the file just fine (however it comes up with a class not found afterwards). I have tried removing the dependencies one by one but each one triggers the error.

Any ideas on how to solve this?

Class not found error

org/openqa/selenium/WebDriver: java.lang.NoClassDefFoundError
java.lang.NoClassDefFoundError: org/openqa/selenium/WebDriver

lambda cannot zip error

Calling the invoke API action failed with this message: Lambda was not able to unzip the file

The dependencies causing the issue

    <dependency>
        <groupId>org.seleniumhq.webdriver</groupId>
        <artifactId>webdriver-common</artifactId>
        <version>0.9.7376</version>
    </dependency>

    <dependency>
        <groupId>org.seleniumhq.selenium</groupId>
        <artifactId>selenium-chrome-driver</artifactId>
        <version>3.141.59</version>
    </dependency>

    <dependency>
        <groupId>org.seleniumhq.selenium</groupId>
        <artifactId>selenium-java</artifactId>
        <version>3.141.59</version>
    </dependency>

updated dependancies (for Vishal)

    <dependency>
        <groupId>org.seleniumhq.webdriver</groupId>
        <artifactId>webdriver-common</artifactId>
        <version>0.9.7376</version>
    </dependency>

    <dependency>
        <groupId>org.seleniumhq.selenium</groupId>
        <artifactId>selenium-chrome-driver</artifactId>
        <version>3.141.59</version>
    </dependency>

    <dependency>
        <groupId>org.seleniumhq.selenium</groupId>
        <artifactId>selenium-java</artifactId>
        <version>3.141.59</version>
    </dependency>

    <dependency>
        <groupId>org.seleniumhq.selenium</groupId>
        <artifactId>selenium-api</artifactId>
        <version>2.0rc2</version>
    </dependency>
    <dependency>
        <groupId>org.seleniumhq.selenium</groupId>
        <artifactId>selenium-remote-driver</artifactId>
        <version>3.141.59</version>
    </dependency>

    <dependency>
        <groupId>org.seleniumhq.selenium</groupId>
        <artifactId>selenium-support</artifactId>
        <version>3.141.59</version>
    </dependency>

Configuration

 <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>3.6.0</version>
        <configuration>
          <source>1.8</source>
          <target>1.8</target>
          <encoding>UTF-8</encoding>
          <forceJavacCompilerUse>true</forceJavacCompilerUse>
        </configuration>
      </plugin>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-shade-plugin</artifactId>
        <version>3.0.0</version>
        <executions>
          <execution>
            <phase>package</phase>
            <goals>
              <goal>shade</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>

I have now added the jars as separate dependencies and still getting the issue — Meerfall the dewott, Oct 22 '19 at 08:25
I think the chromedriver dependency is messing it up. Can you try without the chrome driver once? If it works then try to change the version of the chrome driver dependency — Vishal, Oct 23 '19 at 02:53
@Vishal it still comes up with the unzip error with or without the chrome dependancy — Meerfall the dewott, Oct 23 '19 at 10:03
Try including this in your pom: [Selenium-api](https://mvnrepository.com/artifact/org.seleniumhq.selenium/selenium-api/2.0rc2) — Vishal, Oct 23 '19 at 19:17
You will likely also need: selenium-remote-driver-.jar and probably also selenium-support-.jar, in addition to the selenium-java-.jar you already have. But try with the above one first. As there are transitive dependencies for selenium. Also Make sure you do a **mvn install** before running your lambda — Vishal, Oct 23 '19 at 19:17

score 1 · Answer 1 · answered Oct 25 '19 at 14:51

1

Try to tell your dependencies to output zip, maybe jar is messing up with things

Add this to maven-assembly-plugin configuration:

     <formats>
        <format>zip</format>
     </formats>

For example:

<plugin>
     <groupId>org.apache.maven.plugins</groupId>
     <artifactId>maven-assembly-plugin</artifactId>
     ...
     <configuration>
         ...
         <formats>
            <format>zip</format>
         </formats>
     </configuration>
</plugin>

Same is suggested here

answered Oct 25 '19 at 14:51

Carlos Robles

10,828
3
41
60

Could you give more detail using the pom provided in my question as i'm using the shade plugin not assembly – Meerfall the dewott Oct 25 '19 at 14:59
Is there any reason why you need to use shade? Shade is meant to create an uber-jar, so maybe it will hard to manage. I think it should be easy to use assembly instead and make it work. Im sorry I dont have any lambda ready with maven now so I cannot test myself :/ – Carlos Robles Oct 25 '19 at 15:16
When creating the jar I followed AWS official walkthrough on setting it up and its suppose to save space i think, however if it will work without it then I don't need it. Could you please update your answer with how I should format it so I can test your answer? – Meerfall the dewott Oct 28 '19 at 08:15

score 1 · Answer 2 · answered Oct 31 '19 at 16:30

1

I figured it out. The java selenium seemed to cause the major issue. Downgrading to 3.10 fixed the issue although I have no idea why.

answered Oct 31 '19 at 16:30

Meerfall the dewott

209
1
4
19

Jason Warner · Accepted Answer · 2019-11-01T10:28:33.043

The shade plugin combines all dependencies with the developed code and plops them in one Uber JAR. The downside is that it can overwrite resource files, and doesn't play well with signed jars (in my experience at least).

I would recommend moving away from the shade plugin if at all possible.

That said, if you have to use it - you're issue may be with the combining the jar resources. There are many transformers that you can use to resolve this, and you'll need to investigate which one is really needed. I would start with something like this

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-shade-plugin</artifactId>
    <version>2.4.3</version>
    <configuration>
        <shadedArtifactAttached>true</shadedArtifactAttached>
        <shadedClassifierName>${executable.classifier}</shadedClassifierName>
        <filters>
            <filter>
                <artifact>*:*</artifact>
                <excludes>
                    <exclude>META-INF/*.SF</exclude>
                    <exclude>META-INF/*.DSA</exclude>
                    <exclude>META-INF/*.RSA</exclude>
                </excludes>
            </filter>
        </filters>
    </configuration>
    <executions>
        <execution>
            <phase>package</phase>
            <goals>
                <goal>shade</goal>
            </goals>
            <configuration>
                <transformers>
                    <transformer
                        implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer" />
                    <transformer
                        implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
                        <mainClass>fully.qualified.ClassName</mainClass>
                    </transformer>
                </transformers>
            </configuration>
        </execution>
    </executions>
</plugin>

You can find more tranformers on the Apache plugin here

The alternative that I would suggest is Spring Boot, which uses the Jar-in-Jar structure with a custom ClassLoader to load classes from the internal jar(s).

This is the easier method due to not having to re-write files as the Shade plugin approach and it handles the dependencies a little better.

<plugin>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-maven-plugin</artifactId>
    <version>1.3.6.RELEASE</version>
    <configuration>
        <classifier>${executable.classifier}</classifier>
        <layout>ZIP</layout>
        <mainClass>fully.qualified.ClassName</mainClass>
    </configuration>
    <executions>
        <execution>
            <phase>package</phase>
            <goals>
                <goal>repackage</goal>
            </goals>
        </execution>
    </executions>
</plugin>

Seriously, look at the simpler configuration!

NOTE: Most of this came from my own notes - version numbers may be a little old...

I'm not sure if this is the true answer as I have no need to go with this method (see my answer). I rather stick with the shaded jar since that seems to work. However your answer is detailed and you clearly know what your talking about so i'm going to mark it as correct — Meerfall the dewott, Nov 01 '19 at 08:40

AWS Lambda Jar unable to zip after adding selenium dependencies in pom

3 Answers3

Linked