Limit output with scala on IntelliJ

Question

I am using IntelliJ IDEA for Scala. When I write a block of Scala code, I build the project and execute it. At the bottom window of intellij you can see the code running.

In the output window below I will get a line of output that looks like:

18/03/27 15:07:31 INFO SparkContext: Created broadcast 132 from broadcast at LogisticRegression.scala:1879

All in RED text. There are hundreds of output lines in red text. I am only interested in seeing the white text. For example:

println("My white text")

How do I hide this information overkill in intellij? There is too much info in the output window at the bottom. Where are the options in intellij to curtail what is being output?

I understand this info will be useful if I am debugging and looking for answers as to why something isn't working. But for now, its completely in the way and annoying.

Any help would be great.

Have you tried something like this: https://stackoverflow.com/a/20817292/8230810 — James Whiteley, Mar 27 '18 at 14:17

koiralo · Accepted Answer · 2018-03-27T14:49:37.960

2

Seems like you are you using spark, if so you can set the log level

import org.apache.log4j.Level
import org.apache.log4j.{Level, Logger}

Logger.getLogger("org").setLevel(Level.OFF)
Logger.getLogger("akka").setLevel(Level.OFF)

Or inside your spark folder find conf/log4j.properties and update the log4j.rootCategory to

log4j.rootCategory=INFO, console

Hope this helps

edited Mar 27 '18 at 14:49

answered Mar 27 '18 at 14:19

koiralo

22,594
6
51
72

it's not a great idea to code the logging level into the app as it will require a redeploy to change those settings. – Jason Mar 27 '18 at 14:44
Yeah, that's correct, but to run small spark programs to test from the ide won't have any problems. – koiralo Mar 27 '18 at 14:47
I have found a file called: log4j.properties.template Do I change its name to something else? – JetS79 Mar 27 '18 at 16:02
Yeah change the file name to `log4j.properties` – koiralo Mar 27 '18 at 16:03
Seems to have cut the amount of red messages appearing dramatically. Its a lot more manageable now. I thought the issue was with IntelliJ, but it wasn't. It was obviously feeding the info to spark, and what was appearing in the window was coming from a local install of spark, hence java. – JetS79 Mar 27 '18 at 16:28
yeah that was from the spark log.:) – koiralo Mar 27 '18 at 16:29

score 1 · Answer 2 · answered Mar 27 '18 at 14:19

this is a function of the logging framework being used by the dependencies you've included in your program.

Spark uses log4j to log, details here: https://spark.apache.org/docs/latest/configuration.html#configuring-logging

Essentially, you need to tell the logger what messages to print using the log4j.prooerties file.

Limit output with scala on IntelliJ

2 Answers2