23

Yesterday I faced an interesting issue after deploying my Java 8 webapp on Tomcat 8. Rather than how to solve this issue I'm more interested in understanding why that happens. But let's start from the beginning.

I have two classes defined as follows:

Foo.java

package package1;

abstract class Foo {

    public String getFoo() {
        return "foo";
    }

}

Bar.java

package package1;

public class Bar extends Foo {

    public String getBar() {
        return "bar";
    }

}

As you can see, they are in the same package and, ultimately, end up in the same jar, let's call it commons.jar. This jar is a dependency of my webapp (i.e. as been defined as dependency in my webapp's pom.xml).

In my webapp, there is a piece of code which does:

package package2;

public class Something {

    ...

    Bar[] sortedBars = bars.stream()
                           .sorted(Comparator.comparing(Bar::getBar)
                                             .thenComparing(Bar::getFoo))
                           .toArray(Bar[]::new);

    ...

}

and when it is executed I get:

java.lang.IllegalAccessError: tried to access class package1.Foo from class package2.Something

Playing around and experimenting I was able to avoid the error in three two ways:

  1. changing the Foo class to be public instead of package-private;

  2. changing the package of the Something class to be "package1" (i.e. literally the same as the Foo and Bar classes but physically different being the Something class defined in the webapp);

  3. forcing the class-loading of Foo before executing the offending code:

    try {
        Class<?> fooClass = Class.forName("package1.Foo");
    } catch (ClassNotFoundException e) { }
    

Can someone give me a clear, technical explanation that justifies the issue and the above results?

Update 1

When I tried the third solution I was actually using the commons.jar of the first one (the one where the Foo class is public instead of package private). My bad sorry.

Moreover, as pointed out in one of my comments, I tried to log the classloader of the Bar class and Something class, right before the offending code and the result for both was:

WebappClassLoader
context: my-web-app
delegate: false
----------> Parent Classloader:
java.net.URLClassLoader@681a9515

Update 2

Ok, I finally solved one of the mysteries!

In one of my comments I said that I wasn't able to replicate the problem by executing the offending code from a simple main created in a different package than Foo and Bar of the commons.jar. Well...Eclipse (4.5.2) and Maven (3.3.3) fooled me here!

With this simple pom:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>my.test</groupId>
    <artifactId>commons</artifactId>
    <version>0.0.1-SNAPSHOT</version>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <maven.compiler.source>1.8</maven.compiler.source>
        <maven.compiler.target>1.8</maven.compiler.target>
    </properties>

</project>
  1. if I execute "mvn clean package" (as Eclipse Run Configuration) and run the main from within Eclipse I get the wonderful IllegalAccessError (cool!);

  2. if I execute Maven -> Update project... and run the main from within Eclipse I don't get any error (not cool!).

So I switched to the command-line and I confirmed the first option: the error consistently appears regardless by whether the offending code is in the webapp or in the jar. Nice!

Then, I was able to further simplify the Something class and discovered something interesting:

package package2;

import java.util.stream.Stream;
import package1.Bar;

public class Something {

    public static void main(String[] args) {

        System.out.println(new Bar().getFoo());
        // "foo"

        Stream.of(new Bar()).map(Bar::getFoo).forEach(System.out::println);
        // IllegalAccessError

    }

}

I'm about to be blasphemous here so bear with me: could it be that the Bar::getFoo method reference simply get "resolved" to the Foo::getFoo method reference and, since the Foo class is not visible in Something (being Foo package private), the IllegalAccessError is thrown?

A_Di-Matteo
  • 26,902
  • 7
  • 94
  • 128
fabriziocucci
  • 782
  • 8
  • 20
  • My guess, and it is a guess, is this happens because Foo is package private. So while Bar can be public in the same package, it probably shouldn't be. When you run the web app, Bar is a Foo, and the runtime can't access Foo, so throws your exception. Again a guess but I would guess its the Comparator that is having the issue. I dont understand why the compiler fails to catch this though :S – Gavin Mar 19 '16 at 11:12
  • My intuition too is based on the fact that getFoo is a public method of a package-private class. But then I can't explain why, if I execute the same offending code from another package (e.g. package3) of the commons.jar by simply creating a main, it works. – fabriziocucci Mar 19 '16 at 13:28
  • whoa! this is interesting)) found some old bug reports: https://bugs.eclipse.org/bugs/show_bug.cgi?id=423684 what JDK version you are running? – nukie Mar 19 '16 at 20:44
  • @nukie I also found that bug report, but in this case we are having the opposite: Eclipse is fine, Maven is getting issues – A_Di-Matteo Mar 19 '16 at 20:45
  • More possible workarounds: Override `getFoo()` in `Bar` and delegate to `super.getFoo()`. Declare `getFoo()` in a public interface implemented by `Foo` (or by `Bar`). [Related question](http://stackoverflow.com/q/35548676/521799) – Lukas Eder Mar 21 '16 at 08:00

2 Answers2

20

I was able to reproduce the same issue compiling in Eclipse (Mars, 4.5.1) and from command line using Maven (Maven Compiler Plugin version 3.5.1, the latest at the moment).

  • Compiling and running the main from Eclipse > No Error
  • Compiling from console/Maven and running the main from Eclipse > Error
  • Compiling from console/Maven and running the main via exec:java from console > Error
  • Compiling from Eclipse and running the main via exec:java from console > No Error
  • Compiling from command line directly with javac (no Eclipse, no Maven, jdk-8u73) and running from command line directly with java > Error

    foo
    Exception in thread "main" java.lang.IllegalAccessError: tried to access class com.sample.package1.Foo from class com.sample.package2.Main   
    at com.sample.package2.Main.lambda$MR$main$getFoo$e8593739$1(Main.java:14)   
    at com.sample.package2.Main$$Lambda$1/2055281021.apply(Unknown Source)   
    at java.util.stream.ReferencePipeline$3$1.accept(Unknown Source)   
    at java.util.stream.Streams$StreamBuilderImpl.forEachRemaining(Unknown Source)   
    at java.util.stream.AbstractPipeline.copyInto(Unknown Source)   
    at java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source)   
    at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(Unknown Source)   
    at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(Unknown Source)   
    at java.util.stream.AbstractPipeline.evaluate(Unknown Source)   
    at java.util.stream.ReferencePipeline.forEach(Unknown Source)   
    at com.sample.package2.Main.main(Main.java:14)
    

Note the stacktrace above, the first (pre-java-8) invocation works fine while the second (java-8 based) throws an exception.

After some investigation, I found relevant the following links:

  • JDK-8068152 bug report, describing a similar issue and, above all, mentioning the following concerning the Maven Compiler Plugin and Java:

    This looks like a problem induced by the provided maven plugin. The provided maven plugin (in the "plugin" directory) adds "tools.jar" to the ClassLoader.getSystemClassLoader(), and this is triggering the problem. I don't really see much that could (or should) be done on the javac side, sorry.

    In more details, ToolProvider.getSystemJavaCompiler() will look into ClassLoader.getSystemClassLoader() to find javac classes. If it does not find javac there, it tries to find tools.jar automatically, and creates an URLClassLoader for the tools.jar, loading the javac using this class loader. When compilation runs using this class loader, it loads the classes using this classloader. But then, when the plugins adds tools.jar to the ClassLoader.getSystemClassLoader(), the classes will begin to be loaded by the system classloader. And package-private access is denied when accessing a class from the same package but loaded by a different classloader, leading to the above error. This is made worse by maven caching the outcomes of ToolProvider.getSystemJavaCompiler(), thanks to which running the plugin in between two compilations still leads to the error.

    (NOTE: bold is mine)

  • Maven Compiler Plugin - Using Non-Javac Compilers, describing how you can plug a different compiler to the Maven Compiler Plugin and use it.

So, simply switching from the configuration below:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-compiler-plugin</artifactId>
    <version>3.5.1</version>
    <configuration>
        <source>1.8</source>
        <target>1.8</target>
    </configuration>
</plugin>

To the following:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-compiler-plugin</artifactId>
    <version>3.5.1</version>
    <configuration>
        <source>1.8</source>
        <target>1.8</target>
        <compilerId>eclipse</compilerId>
    </configuration>
    <dependencies>
        <dependency>
            <groupId>org.codehaus.plexus</groupId>
            <artifactId>plexus-compiler-eclipse</artifactId>
            <version>2.7</version>
        </dependency>
    </dependencies>
</plugin>

Fixed the issue, no IllegalAccessError any more, for the same code. But doing so, we actually removed the diff between Maven and Eclipse in this context (making Maven using the Eclipse compiler), so it was kind of normal result.

So indeed, this leads to the following conclusions:

  • The Eclipse Java compiler is different than the Maven Java Compiler, nothing new in this case, but that's yet another confirmation
  • The Maven Java compiler in this case has an issue, while the Eclipse Java compiler has not. The Maven Compiler is coherent with the JDK compiler though. So it might actually be a bug on the JDK having effect on the Maven Compiler.
  • Making Maven using the same Eclipse compiler fixes the issue, or hides it.

For reference, I tried also the following without much success before switching to the eclipse compiler for Maven:

  • Changing Maven Compiler Plugin version, every version from 2.5 till 3.5.1
  • Trying with JDK-8u25, JDK-8u60, JDK-8u73
  • Making sure Eclipse and Maven Compiler were use exactly the same javac, explicitly using the executable option

To summarize, the JDK is coherent with Maven, and it is most probably a bug. Below some related bug reports I found:

  • JDK-8029707: IllegalAccessError using functional consumer calling inherited method. Fixed as Won't Fix (it was exactly the same issue)
  • JDK-8141122: IllegalAccessException using method reference to package-private class via pub. Open (again, exactly the same issue)
  • JDK-8143647: Javac compiles method reference that allows results in an IllegalAccessError. Fixed in Java 9 (similar issue, pre-java-8 code would work fine, java-8 style code would break)
  • JDK-8138667: java.lang.IllegalAccessError: tried to access method (for a protected method). Open (similar issue, compilation fine but than runtime error for illegal access on lambda code).
Community
  • 1
  • 1
A_Di-Matteo
  • 26,902
  • 7
  • 94
  • 128
  • astonishing Maven) – nukie Mar 19 '16 at 20:55
  • Great answer @A. Di Matteo! I did a further experiment by compiling and running from command-line using pure javac and java commands (no Maven, no Eclipse) and...I still see the error. I'm using java 1.8.0_60 on Windows. – fabriziocucci Mar 19 '16 at 22:27
  • @fabriziocucci ops, I tried ealier the same and it was coherent with Eclipse, I tried again right now and indeed it is not, it is coherent with Maven! – A_Di-Matteo Mar 19 '16 at 22:34
  • @fabriziocucci I also found [this bug report](https://bugs.openjdk.java.net/browse/JDK-8029707) for the same which was discarded afterwards though – A_Di-Matteo Mar 19 '16 at 22:44
  • @A. Di Matteo I checked out the bug report you mentioned and it is indeed the same problem. The assignee was not able to reproduce the issue according to that description but in this case it seems indeed reproducible (unless we are both messing up with the command line!). I was trying to understand from the JLS if, according to the evaluation model of the method references, this error is expect or not. – fabriziocucci Mar 19 '16 at 23:07
  • @A. Di Matteo I just filed a bug report for the jdk because I couldn't find anything else in the JLS. Let's see what happens next... – fabriziocucci Mar 20 '16 at 10:36
  • @fabriziocucci I updated my answer according to latest findings, I also found additional **3** existing bug reports for similar cases which make this use case most probably a bug. Feel free to edit the answer adding the link to your bug report as well – A_Di-Matteo Mar 20 '16 at 21:24
  • @A.DiMatteo the JDK-8141122 is exactly our issue so I guess my bug report won't get through! :) Great job in finding the other related bug reports. – fabriziocucci Mar 20 '16 at 22:27
  • 2
    Note that this bug has been fixed in JDK8u102, see also [this answer](http://stackoverflow.com/a/39120603/2711488) – Holger Oct 27 '16 at 12:34
4

If packages commons.jar and jar with package2 are loaded by another class-loader, then it is different runtime packages and this fact preventing methods of Something class from access to package members of Foo. See chapter 5.4.4 of JVM spec and this awesome topic.

I think there is one more solution in addition to what you already tried: override method getFoo in Bar class

Community
  • 1
  • 1
nukie
  • 691
  • 7
  • 14
  • If before executing the offending code I log the classloader of both classes (Bar.class.getClassloader() and Something.class.getClassloader) I get the same result. That would make me think that the classes have indeed been loaded by the same classloader. Is this result misleading? – fabriziocucci Mar 19 '16 at 11:36
  • hm... it was very same class-loader instance? have you checked parent class loader for them? do they share the same class-loading hierarchy? – nukie Mar 19 '16 at 11:44
  • By logging the classloader of both classes I see "WebappClassLoader context: my-web-app delegate: false --------> Parent Classloader: java.net.UrlClassLoader@681a9515". – fabriziocucci Mar 19 '16 at 12:33