19

Background: I am trying to build Hadoop for Windows using instructions found here. I have OpenJDK 11 and have ran into the following issue while running mvn package:

Could not resolve dependencies for project org.apache.hadoop:hadoop-annotations:jar:2.5.0-SNAPSHOT: Could not find artifact jdk.tools:jdk.tools:jar:1.6 at specified path C:\Program Files\Java\jdk-11.0.1\..\lib\tools.jar

I realised no tools.jar exists in the Windows build of OpenJDK 11. Would I have to build this from source as well? How can I get the tools.jar Maven artefact for OpenJDK 11?

Naman
  • 27,789
  • 26
  • 218
  • 353
R Y
  • 455
  • 1
  • 3
  • 13
  • @nullpointer I've confirmed in my sandboxes that Hadoop 2.9.2 will work with JDK 1.8, and I'm using the OpenJDK release. That said, JEP 220's improvements will likely keep any Hadoop 2.x series from running on Java 9. – Edwin Buck Dec 10 '18 at 14:37
  • 1
    A better link to track the progress is [Hadoop and Java 9, 10, 11](https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+and+Java+9%2C+10%2C+11), since they're targetting to compatible with the LTS version rather than 9 and 10 in short term. – Naman Dec 10 '18 at 14:44
  • Basically all deps that were on `tools.jar` now are on jdk classpath by default, then you don't need to import it. The problem is hadoop expecting to import the jar – deFreitas Feb 24 '19 at 18:59

1 Answers1

18

It was removed in JDK 9.

JEP 220 removed the well-known and frequently-referenced JARs rt.jar and tools.jar. According to its description, "The class and resource files previously stored in lib/rt.jar, lib/tools.jar, lib/dt.jar, and various other internal jar files will now be stored in a more efficient format in implementation-specific files in the lib directory.

I suggest you use a updated version of Apache Hadoop. If the updated version doesn't support Java 11 either, downgrade your Java to 1.8.

For example, the current stable Apache Hadoop is version 2.9.2, and in its instructions on how to install, it specifically references JDKs before JDK 9. For its proper support, I suggest you use OpenJDK 1.8

There is a Hadoop 3.1.1, and if you find the Java version is critical to your success, I'd look into that.

Paulo Merson
  • 13,270
  • 8
  • 79
  • 72
Edwin Buck
  • 69,361
  • 7
  • 100
  • 138
  • *whatever was looking for tools.jar"* !! can you state a way to figure out what exactly is it? – Naman Dec 10 '18 at 14:30
  • 1
    Apache Hadoop was looking for tools.jar. You're using a JVM that was released after your version of Apache Hadoop was initially released, even if it has had maintenance (no new features) releases afterwards. You're not going to get the old software to match the new patterns without someone rewriting it. Odds are your choices are to upgrade Apache Hadoop to 3.1.1 (and hope that's new enough to support JDK 11) or to install and use Java 8. – Edwin Buck Dec 10 '18 at 14:35
  • *Apache Hadoop was looking for tools.jar.* I doubt that.... *hope that's new enough to support JDK 11* and sorry this doesn't answer the question – Naman Dec 10 '18 at 14:49
  • 2
    @nullpointer I'm not sure, but I'm willing to believe you. Hadoop has tons of components, and while I am certain that HDFS, and MapReduce likely don't need it, I am not going to do a full survey of the suite. What I do know is that you're not getting a tools.jar in a JDK 1.11. So, either way, he'll have to adjust to JDK 1.8. Of course, if you think Hadoop doesn't need it, then by all means keep running Hadoop on 1.11, and tell me how you resolve the tools.jar. I'd be interested. – Edwin Buck Dec 10 '18 at 14:53
  • *I do know is that you're not getting a tools.jar* ... agreed ...correction JDK 11 and not 1.11.. *I'd be interested.* that's what I was interested in thinking you were posting an answer. Don't take it wrong, but I was actually looking for an answer and not revert to JDK8 as an answer. – Naman Dec 10 '18 at 14:56
  • 2
    @nullpointer Fixing the problem is removing the assumptions from the software looking for a file called tools.jar, and instead relying on the already existing JAR files in the JVM to provide the needed functionality. As he's told me he's using "hadoop" whatever he's using, he's self identified it as "hadoop" and he should look into whatever he's calling "hadoop". If you think you can patch a JDK to have a tools.jar after it has been removed, you can. It's just not a supportable solution, as you'd have to deploy a forked version of the JVM / JDK (and couldn't call it Java). – Edwin Buck Dec 10 '18 at 14:59