how can I access spark javadoc or sources from java project?

Question

I'm using Spark from a java project on IntelliJ 14 CE. Is there any way to navigate to the Spark sources or javadoc? By default it only shows the rough decompile code without any comment. I wouldn't mind navigating to the scala code if there was a way to do it. But maybe it would be better to plug in the javadoc, but I cannot find it anywhere

thanks

I guess I am understanding it incorrectly but Right Mouse Click Go To -> Declaration should do the trick? — mr. Holiday, Aug 18 '15 at 08:25
@IliaAptsiauri this takes you only to the decompiled code without javadoc — Daniel Pinyol, Aug 18 '15 at 08:28
afterwards you can press F1. "IntelliJ IDEA recognizes inline documentation created in accordance with Javadoc markup." — mr. Holiday, Aug 18 '15 at 08:32
@IliaAptsiauri also tried it. comments do not appear because the bytecode jar does not have any. Comments are available only in scala source code (which intellij does not read by default), or at http://spark.apache.org/docs/latest/api/java/. — Daniel Pinyol, Aug 18 '15 at 08:37
Ok gotcha now, the last try :D, I guess you can attach the documentation by yourself check http://stackoverflow.com/questions/91307/attaching-additional-javadoc-in-intellij-idea out might help — mr. Holiday, Aug 18 '15 at 08:44
@IliaAptsiauri After attaching https://spark.apache.org/docs/latest/api/java/ to each spark dependency, F1 works. Thank you! Go ahead and add it as and answer. Strangely F1 does not work from the decompiled code — Daniel Pinyol, Aug 18 '15 at 09:10
Feel free to retweet https://twitter.com/h_schreiber/status/823789681330503680 — Hendrik, Jan 24 '17 at 12:16

score 7 · Answer 1 · answered Jul 01 '18 at 20:39

7

What did the trick to me was to install the Scala plugin for IntelliJ, I was then able to navigate and debug through the spark core (which is written in scala), in spite of the project being in Java, and properly see the Javadoc, which was automatically inferred from the scala source code. Of course, you also need to have the Spark source code properly setup, easy to do using Gradle or Maven.

answered Jul 01 '18 at 20:39

FelipeKunzler

1,204
1
11
17

1

Thnx! Worked for me. – Artemiy Aug 22 '18 at 10:16
1

@FelipeKunzler you're a lifesaver! – Jewels May 27 '19 at 08:02

score 2 · Accepted Answer · edited May 23 '17 at 12:09

After discussion in the comments the solution appears to be to just add manually the JavaDoc to desired "Module" or Project.

The detailed answer is located at the following thread Attaching additional javadoc in Intellij IDEA

Then select the "Dependencies" tab, select the dependency that's missing the javadoc and click "Edit". In the window that just showed up you see two buttons "Attach Javadoc" and "Specify Javadoc URL". If you have the javadoc in a jar file select the first one, if you want to point to a web site that contains the javadoc select the latest. That's it.

score 1 · Answer 3 · edited Nov 12 '15 at 19:40

1

In your sbt build file you can always do this:

"org.apache.spark" %% "spark-core" % versions('spark) withSources() withJavadoc(),
"org.apache.spark" %% "spark-sql" % versions('spark) withSources() withJavadoc(),

versions('spark) is just the version of spark that you are using.

edited Nov 12 '15 at 19:40

ndmeiri

4,979
12
37
45

answered Nov 12 '15 at 18:44

M.Rez

1,802
2
21
30

score 1 · Answer 4 · answered Mar 26 '19 at 00:21

"right click on your pom.xml" -> "Maven" -> then choose "Download Sources and Documentation"

To avoid this in the future: "Preferences" -> "Build, Execution, Deployment" -> "Build Tools" -> "Maven" -> "Importing" -> Check the Automatically download Sources and Documentation boxes.

Credit to Yacov Schondorf and Stephen Boesch from Intellij user forums.

how can I access spark javadoc or sources from java project?

4 Answers4