0

I have a question inlog4j FileAppender

/home/omega/conf/log4j.properties :

log4j.rootCategory=INFO, file    

# Direct log messages to a log file
log4j.appender.file=org.apache.log4j.RollingFileAppender
log4j.appender.file.File=../logs/omega.log
log4j.appender.file.DatePattern='.'yyyy-MM-dd-HH-mm
log4j.appender.file.MaxFileSize=300MB
log4j.appender.file.MaxBackupIndex=10
log4j.appender.file.layout=org.apache.log4j.PatternLayout
log4j.appender.file.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p %c{1}: %m%n

Spark-Submit Command :

spark-submit --class ${MAIN_CLASS} \
--driver-java-options "-Dlog4j.configuration=file:/home/omega/conf/log4j.properties" \
--master ${MASTER_URL} \
--num-executors ${NUM_EXECUTORS} \
--executor-memory ${EXECUTOR_MEMORY} \
--executor-cores ${EXECUTOR_CORES} \
 ${APP_JAR} ${CONF_PATH} ${INPUT_FILE_NAME}  ${FILE_DATE}

I see that log file has been created as omega.log for my First run on spark submit command

But when I run the spark submit command for the second time, i see that same log file is being appended with news logs..

but I want a new log file to created every time i run my spark submit command and also i want the tiemstamp to be added to my log file name

Basically I want a new log file to be created with timestamp in its name

I tied to achieve this with DailyRollingFileAppender , but it creates more log files for a single run.

So expected log files :

omega_timestamp_for_first_run.log 
omega_timestamp_for_second_run.log
omega_timestamp_for_third_run.log

Actually I am looking for something like this

 ex : omega_2017_05_09_11_05.log

Could someone help me on this?

Surender Raja
  • 3,553
  • 8
  • 44
  • 80
  • this link can be helpful. http://stackoverflow.com/questions/31360185/creating-log4j-log-file-for-each-run-with-date-and-timestamp http://stackoverflow.com/questions/19132433/one-logfile-per-run-with-log4j – Sandeep Singh May 09 '17 at 11:06
  • @Sandeep : Yes. That Link was very helpful. Thanks Much for sharing that – Surender Raja May 11 '17 at 09:14

0 Answers0