1

I am experimenting with apache spark 3 in intellij by creating a simple standalone scala application. When I run my program I get lots of INFO logs. Based on various SO answers I tried all of the following:

spark.sparkContext.setLogLevel("ERROR")
SparkSession.builder.getOrCreate().sparkContext.setLogLevel("ERROR")
Logger.getRootLogger().setLevel(Level.ERROR)
Logger.getLogger(classOf[RackResolver]).getLevel
Logger.getLogger("org").setLevel(Level.OFF)
Logger.getLogger("akka").setLevel(Level.OFF)

But none works. I still see tons of INFO logs.

So what's the working way of achieving this?

Mandroid
  • 6,200
  • 12
  • 64
  • 134

0 Answers0