0

I am doing a project with spark graphx. i want to log on my codes for tracing, ... i use the below codes for creating logger object.

package org.araz.DAF.GraphAnalysis
import org.apache.spark.graphx._
import org.araz.DAF.{GEdge, GVertex}
import org.slf4j.LoggerFactory
class GraphAnalysis(_graph: Graph[GVertex, GEdge]) {
  var graph: Graph[GVertex, GEdge] = _graph
  def GetConnectedComponents(_cnt: Int): Array[ConnectedComponents] = {
    val logger= LoggerFactory.getLogger(getClass)
    val cc = graph.connectedComponents()
    logger.info("Info 1")
    cc
  }
}

when i open the log file, i found more lines rather than "Info 1", more lines of graphx library. spark inner logs. how can i log only my logs and ignore outer libraries log

user10938362
  • 3,991
  • 2
  • 12
  • 29
user2352554
  • 521
  • 4
  • 17
  • Did you try: https://stackoverflow.com/questions/27781187/how-to-stop-info-messages-displaying-on-spark-console ? – Belwal Mar 22 '20 at 18:30
  • thanks 4 you comment. but that link ignores WARN logs if i log from WARN type. so my logs missed. – user2352554 Mar 22 '20 at 18:34

1 Answers1

0

Update Log4j properties file:

log4j.rootLogger=ERROR
...
...
...
log4j.logger.org.araz.DAF.GraphAnalysis=INFO

Easiest way to stop printing spark logs could be:

Logger.getLogger("org").setLevel(Level.ERROR)// OFF
Logger.getLogger("akka").setLevel(Level.ERROR)//OFF
Belwal
  • 453
  • 1
  • 4
  • 15