0

I made a pretty standard log4j file, I would like to load and use the properties of it, with it being packed inside the application jar.

Using the Properties from java.util I was able to get it working while launching in yarn-client mode, but using typesafe-config, at the moment, the log4j.properties is completely ignored.

This is the log4j.properties

log4j.rootLogger=INFO, RollingAppender
log4j.rootCategory=DEBUG, file
log4j.appender.RollingAppender=org.apache.log4j.DailyRollingFileAppender
log4j.appender.RollingAppender.File=log/rule.log
log4j.appender.RollingAppender.DatePattern='.'yyyy-MM-dd
log4j.appender.RollingAppender.layout.ConversionPattern=[%p] %d %c %M - %m%n
log4j.appender.RollingAppender.layout=org.apache.log4j.PatternLayout



log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p %c{1} %m%n

This is how I retrieve the properties. This should be fine, since the other properties are retrieved correctly.

val log = LogManager.getRootLogger()
val logCfgProps : Config =  ConfigFactory.load("log4j.properties")
val configLogProps = Option(getClass.getClassLoader.getResource("log4j.properties")).fold
{
  log.error("log4j.properties file not retrieved. Application is shutting down")
  ConfigFactory.load()
}
{ resource => ConfigFactory.load(logCfgProps)}

And I try to log with

log.error("blabla")

On the console I get the right logging but it is not written to file. I tried to add the option --conf "spark.executor.extraJavaOptions=-Dlog4j.configuration=log4j.properties" myapp.jar suggested here but it does not make any difference.

EDIT

I opened a new question on SO on how to use Property configurator to make it work, I leave this open since it was about using ConfigFactory.

Community
  • 1
  • 1
Irene
  • 744
  • 1
  • 12
  • 36
  • 1
    Could you check the file name in jar? Is it `log4j-executor.properties` or `log4j.properties`? I'm confused because you used two different file names in your question. – zsxwing Sep 18 '15 at 15:48

1 Answers1

1

Your code just reads the properties into the Config object, but this doesn't affect log4j, which has no idea about Typesafe Config. It should read the properties file and use it on its own. You should:

  1. Check that log4j.properties is in src/main/resources and added to the jar correctly;
  2. If it is, but the logging doesn't work, pass -Dlog4j.debug in spark.executor.extraJavaOptions to see what's happening during initialization;
  3. You can also use PropertyConfigurator explicitly by calling one of its configure methods.
Alexey Romanov
  • 167,066
  • 35
  • 309
  • 487