The intersection of Java generics and Scala is...not going well

Question

I freely admit to being a little out of my depth here. My formal training in type systems is a good few decades behind me. I’ve used generics in Java rather trivially once or twice, but they’re not something about which I can claim to have a deep and thorough understanding. I’m also a relative newcomer to Scala, so I’m not claiming a deep or thorough understanding of its type system either.

I set out to update my XML Calabash v2 implementation, written in Scala (2.12 today) to use Saxon 9.9. Saxon 9.9 introduces generics in a number of places. Fine by me. I can cope, I imagine.

Except, I can’t apparently.

The stumbling block is trying to implement a class that extends the ExtensionFunctionDefinition class. It has an inner class that extends the ExtensionFunctionCall class. That, in turn, has an abstract method, call, defined thusly in Java:

public abstract Sequence<?> call(
    XPathContext context,
    Sequence[] arguments
)

My first attempt to define this in Scala was:

override def call(
    context: XPathContext,
    arguments: Array[Sequence]
): Sequence[_]

but that doesn’t compile: “trait Sequence takes type parameters”.

Which is true:

public interface Sequence<T extends Item<?>>

(Item, btw, is:

public interface Item<T extends Item<?>>
extends GroundedValue<T>

which I find slightly confusing for other reasons)

For my second attempt, I tried:

override def call(
    context: XPathContext,
    arguments: Array[Sequence[_]]
): Sequence[_]

But that, I’m told, doesn’t override anything. Hark, the compiler says:

[error] (Note that Array[net.sf.saxon.om.Sequence]
does not match Array[net.sf.saxon.om.Sequence[_]]:
their type parameters differ)

And here we seem to be at an impasse. I can just implement the damned thing in Java, of course, but is this an actual limitation in Scala or in my understanding?

I was lying before, by the way, about my first attempt. My first attempt was actually:

override def call(
    context: XPathContext,
    arguments: Array[Sequence[_ <: Item[_ <: Item[_]]]]
): Sequence[_ <: Item[_ <: Item[_]]]

which I crafted by bluntly copying Java into Scala and letting IntelliJ IDEA translate it. I had failed to work out what to do with the recursive nature of the Item declaration.

Another question about [raw types](https://stackoverflow.com/questions/55514277/why-is-collection-not-simply-treated-as-collection) is currently in the Hot Questions. What a strange coincidence. — Andrey Tyukin, Apr 04 '19 at 23:44

score 3 · Answer 1 · answered Apr 04 '19 at 18:48

3

Try

override def call(context: XPathContext, arguments: Array[Sequence[_ <: Item[_]]]): Sequence[_] = ???

answered Apr 04 '19 at 18:48

Dmytro Mitin

48,194
3
28
66

That's unsuccessful: (Note that Array[net.sf.saxon.om.Sequence[_]] does not match Array[net.sf.saxon.om.Sequence[_ <: net.sf.saxon.om.Item[_]]]: their type parameters differ) – Norm Apr 04 '19 at 21:52
@Norm it compiles https://gist.github.com/DmytroMitin/7d7e67274061f90572d63672e67e09e6 – Dmytro Mitin Apr 04 '19 at 22:28
@Norm in some sense they do match. You can check that `implicitly[Array[Sequence[_ <: Item[_]]] =:= Array[Sequence[_]]]` `implicitly[Array[Sequence[_]] =:= Array[Sequence[_ <: Item[_]]]]` compile. – Dmytro Mitin Apr 04 '19 at 22:36

Andrey Tyukin · Answer 2 · 2019-04-04T22:26:52.497

This here definitely compiles (and thereby confirms that Dmytro Mitin's proposal works):

// ExtensionFunctionCall.java
public interface ExtensionFunctionCall {
  Sequence<?> call(String ctx, Sequence[] args);
}

// Item.java
public interface Item<T extends Item<?>> {}

// Sequence.java
public interface Sequence<T extends Item<?>> {}

// Impl.scala
class Impl extends ExtensionFunctionCall {
  override def call(
    ctx: String,
    args: Array[Sequence[_ <: Item[_]]]
  ): Sequence[_] = ???
}

By the way, it's not just Scala's problem. If you forget Scala for a second, and try to implement it in Java, you get essentially the same errors:

class ImplJava implements ExtensionFunctionCall {
  public Sequence<?> call(
    String ctx,
    Sequence<?>[] args
  ) {
    return null;
  }
}

gives:

ImplJava.java:1: error: ImplJava is not abstract and does not override abstract method call(String,Sequence[]) in ExtensionFunctionCall
class ImplJava implements ExtensionFunctionCall {
^
ImplJava.java:2: error: name clash: call(String,Sequence<?>[]) in ImplJava and call(String,Sequence[]) in ExtensionFunctionCall have the same erasure, yet neither overrides the other
  public Sequence<?> call(
                     ^
2 errors

Now, this is really mystifying, I have no idea how to write down this type in Java. I'm not sure whether it's even expressible in Java without reverting to 1.4-style. The Sequence[] thing is just evil, or, to quote this wonderful article linked by Dmytro Mitin:

Raw Types are bad. Stop using them

For context, this works fine in Java in my context: public Sequence> call(XPathContext context, Sequence[] arguments) — Norm, Apr 04 '19 at 21:57
@Norm My problem is that there are seemingly two separate concepts: `Sequence` and `Sequence>`, which are completely incompatible and not mutually interchangeable, and which are both mapped to Scala's existential types in some strange way. I wasn't even aware that there were *two* families of Java types which are both mapped to Scala's existentials. I was certain that after Java 1.4 the old erased generics became mere syntactic sugar for something that is expressible by wildcards, but this is not the case, apparently. — Andrey Tyukin, Apr 04 '19 at 22:14
Fair enough. Thanks for the very handy link. I have no control over the Java library, though I've passed the link along. In the meantime, I wrote some Java shim code. — Norm, Apr 05 '19 at 01:32
@AndreyTyukin You can avoid raw types not always. For example because of higher-kinded types in Scala absent in Java https://stackoverflow.com/questions/55528032/how-to-export-scala-transformation-to-java/ — Dmytro Mitin, Apr 05 '19 at 07:25

score 0 · Answer 3 · answered Apr 04 '19 at 19:39

0

I think Sequence with no type parameters in java translates into Sequence[Foo] where Foo is the highest possible super-type (Item in this case). So, I would expect something like this to work:

override def call(context: XPathContext, arguments: Array[Sequence[Item[_]]]): Sequence[_] = ???

answered Apr 04 '19 at 19:39

Dima

39,570
6
44
70

No, this doesn't compile: `Error:(6, 14) method call overrides nothing. Note: the super classes of object App contain the following, non final members named call: def call(x$1: com.sun.org.apache.xpath.internal.XPathContext,x$2: Array[App_1.Sequence]): App_1.Sequence[_] override def call(context: XPathContext, arguments: Array[Sequence[Item[_]]]): Sequence[_] = ???`. – Dmytro Mitin Apr 04 '19 at 19:46
Yep, this one appears not to override either of the definitions of call. – Norm Apr 04 '19 at 21:54

The intersection of Java generics and Scala is...not going well

3 Answers3