scalap and javap showing different method signature

Question

I am facing very weird issue, i am having a maven dependency in my project

<dependency>
  <groupId>org.apache.spark</groupId>
  <artifactId>spark-core_2.11</artifactId>
  <version>2.0.0</version>
</dependency>

And this is the class org.apache.spark.ui.WebUI which i am using from this dependency, and it has one method with below signature

def getHandlers : scala.Seq[org.eclipse.jetty.servlet.ServletContextHandler] = { /* compiled code */ }

but when i am executing my program, its failing with exception NoSuchMethodError.

So i started little digging it. when i decompile my jar with scalap its shows me above signature, but when i decompile with javap option, it shows different.

public scala.collection.Seq<org.spark_project.jetty.servlet.ServletContextHandler> getHandlers();

Answer 1

scala.Seq is a type alias for scala.collection.Seq (this changed in Scala 2.13)

https://github.com/scala/scala/blob/v2.11.12/src/library/scala/package.scala#L45

type Seq[+A] = scala.collection.Seq[A]

org.spark_project.jetty is instead of org.eclipse.jetty because of shading

https://github.com/apache/spark/blob/v2.0.0/pom.xml#L2239-L2285

    <!-- Package to use when relocating shaded classes. -->
    <spark.shade.packageName>org.spark_project</spark.shade.packageName>
    ...

      <!--
        The shade plug-in is used here to create effective pom's (see SPARK-3812), and also
        remove references from the shaded libraries from artifacts published by Spark.
      -->
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-shade-plugin</artifactId>
        <configuration>
          <shadedArtifactAttached>false</shadedArtifactAttached>
          <artifactSet>
            <includes>
              <include>org.spark-project.spark:unused</include>
              <include>org.eclipse.jetty:jetty-io</include>
              <include>org.eclipse.jetty:jetty-http</include>
              <include>org.eclipse.jetty:jetty-continuation</include>
              <include>org.eclipse.jetty:jetty-servlet</include>
              <include>org.eclipse.jetty:jetty-servlets</include>
              <include>org.eclipse.jetty:jetty-plus</include>
              <include>org.eclipse.jetty:jetty-security</include>
              <include>org.eclipse.jetty:jetty-util</include>
              <include>org.eclipse.jetty:jetty-server</include>
              <include>com.google.guava:guava</include>
            </includes>
          </artifactSet>
          <relocations>
            <relocation>
              <pattern>org.eclipse.jetty</pattern>
              <shadedPattern>${spark.shade.packageName}.jetty</shadedPattern>
              <includes>
                <include>org.eclipse.jetty.**</include>
              </includes>
            </relocation>
            <relocation>
              <pattern>com.google.common</pattern>
              <shadedPattern>${spark.shade.packageName}.guava</shadedPattern>
            </relocation>
          </relocations>
        </configuration>
        <executions>
          <execution>
            <phase>package</phase>
            <goals>
              <goal>shade</goal>
            </goals>
          </execution>
        </executions>
      </plugin>

What is an uber jar?

What is a shaded jar? And what is the difference/similarities between uber jar and shaded jar?

What is the maven-shade-plugin used for, and why would you want to relocate Java packages?

How maven shading works?

How the class loader works in Java with a shaded jar

---

Here is one more case where scalap and javap outputs differ

Run a scala code jar appear NoSuchMethodError:scala.Predef$.refArrayOps

scala-2.13.10/bin/scalap -cp scala-2.13.10/lib/scala-library.jar scala.Predef$

def refArrayOps(scala.Array[scala.Any]): scala.Any

jdk-17.0.5/bin/javap -cp scala-2.13.10/lib/scala-library.jar scala.Predef$

public <T> T[] refArrayOps(T[])

javap shows the return type of refArrayOps as T[] i.e. Array[T] or Array[Any] while scalap shows just Any. This is because of @inline and AnyVal

https://github.com/scala/scala/blob/v2.13.10/src/library/scala/Predef.scala#L464

https://github.com/scala/scala/blob/v2.13.10/src/library/scala/collection/ArrayOps.scala#L197

@inline implicit def refArrayOps[T <: AnyRef](xs: Array[T]): ArrayOps[T] = new ArrayOps(xs)

final class ArrayOps[A](private val xs: Array[A]) extends AnyVal { ... }

@inline https://www.scala-lang.org/api/2.13.10/scala/inline.html

extends AnyVal https://docs.scala-lang.org/overviews/core/value-classes.html