-1

I have created an application using log4j for storing all the level based logs in a single file, but in the current case the single file will hold all the level logs, such as ALL, INFO, ERROR, FATAL, WARN, DEBUG etc... since in my application i am having an option say a select box upon which when i select ERROR, then it should show only error logs, if i select INFO then it should only INFO logs.

I am parsing the logs file to get the appropriate logs based on a particular level, but in this case say if the log file is very big then it will take more time.

One solution which i have thought for this is to maintain separate log files for each levels, so that we can avoid parsings

Like to know everyone's thought on this. Still if this is fine how can we redirect level based logs to separate log file like ERROR logs to appError.log, INFO logs to appInfo.log, and a common log file to hold all the level based logs.

My log4j.properties is as given below

# LOG4J configuration
log4j.rootLogger=DEBUG, Appender1, Appender2

log4j.appender.Appender1=org.apache.log4j.ConsoleAppender
log4j.appender.Appender1.layout=org.apache.log4j.PatternLayout
log4j.appender.Appender1.layout.ConversionPattern=%d [%t] %-7p %10c{1} - %m%n

log4j.appender.Appender2=org.apache.log4j.FileAppender
log4j.appender.Appender2.File=${appRootPath}/WEB-INF/logs/LogDemos.log
log4j.appender.Appender2.layout=org.apache.log4j.PatternLayout
log4j.appender.Appender2.layout.ConversionPattern=%d [%t] %-7p %10c{1} - %m%n

The parsing method is like as shown below

Scanner scanner = new Scanner(new File(ctx.getRealPath("/")+"/WEB-INF/logs/LogDemos.log"));
        while(scanner.hasNext())
        {
            String level = request.getParameter("level");
            String logs = scanner.nextLine();

            String regex = "(\\d{4}-\\d{2}-\\d{2}) (\\d{2}:\\d{2}:\\d{2},\\d{3}) \\[(.*)\\] ([^ ]*) +([^ ]*) - (.*)$";

            Pattern p = Pattern.compile(regex);

            Matcher m = p.matcher(logs);

            if (m.matches() && m.groupCount() == 6) {

                String date = m.group(1);
                String time = m.group(2);
                String threadId = m.group(3);
                String priority = m.group(4);
                String category = m.group(5);
                String message = m.group(6);
                if(priority.trim().equalsIgnoreCase(level.trim()))
                {
                    logsList.add(logs);
                    System.out.println("date: " + date);
                    System.out.println("time: " + time);
                    System.out.println("threadId: " + threadId);
                    System.out.println("priority: " + priority);
                    System.out.println("category: " + category);
                    System.out.println("message: " + message);
                }
                else
                {
                    logsList.add(logs);
                }
            }

        }
        session.setAttribute("logs", logsList);
        scanner.close();
Alex Man
  • 4,746
  • 17
  • 93
  • 178
  • Have you tried using Splunk or Apache Spark to parse and analyze the log? – Boris Pavlović Jan 22 '15 at 13:21
  • Closely related to [how to get error messages to stderr and informational messages to stdout](http://stackoverflow.com/questions/8489551/logging-error-to-stderr-and-debug-info-to-stdout-with-log4j). – Raedwald Jan 22 '15 at 13:23
  • @BorisPavlović no...i am using simple java with regular expression for paring log file,...i have updated my question with that code – Alex Man Jan 22 '15 at 13:24
  • possible duplicate of [log different log levels to different files with log4j](http://stackoverflow.com/questions/12459943/log-different-log-levels-to-different-files-with-log4j) – Raedwald Jan 22 '15 at 13:24
  • @AlexMan yes, you are right, but maybe some other already existing tool would give you some better results... – Boris Pavlović Jan 22 '15 at 13:29
  • @BorisPavlović can you please share me how to parse log4j logs using either Splunk or Apache Spark – Alex Man Jan 22 '15 at 14:18

2 Answers2

3

If you use org.apache.log4j.varia.LevelRangeFilter, you can do as follows (for INFO and ERROR):

# LOG4J configuration
log4j.rootLogger=DEBUG, OnlyInfo, OnlyError

# Only INFO
log4j.appender.OnlyInfo=org.apache.log4j.FileAppender
log4j.appender.OnlyInfo.File=${appRootPath}/WEB-INF/logs/info.log
log4j.appender.OnlyInfo.File=logs/info.log
log4j.appender.OnlyInfo.layout=org.apache.log4j.PatternLayout
log4j.appender.OnlyInfo.layout.ConversionPattern=%d [%t] %-7p %10c{1} - %m%n
log4j.appender.OnlyInfo.filter.A=org.apache.log4j.varia.LevelRangeFilter
log4j.appender.OnlyInfo.filter.A.LevelMin=INFO    
log4j.appender.OnlyInfo.filter.A.LevelMax=INFO
log4j.appender.OnlyInfo.filter.A.AcceptOnMatch=true

# Only ERROR
log4j.appender.OnlyError=org.apache.log4j.FileAppender
log4j.appender.OnlyError.File=${appRootPath}/WEB-INF/logs/error.log
log4j.appender.OnlyError.layout=org.apache.log4j.PatternLayout
log4j.appender.OnlyError.layout.ConversionPattern=%d [%t] %-7p %10c{1} - %m%n
log4j.appender.OnlyError.filter.A=org.apache.log4j.varia.LevelRangeFilter
log4j.appender.OnlyError.filter.A.LevelMin=ERROR    
log4j.appender.OnlyError.filter.A.LevelMax=ERROR
log4j.appender.OnlyError.filter.A.AcceptOnMatch=true
Paul Vargas
  • 41,222
  • 15
  • 102
  • 148
1

The default assumption that is wired in to Log4J is that whenever you care about messages at a certain level (e.g. INFO) then you also care about more important messages too (e.g. ERROR). You can set levels on loggers and thresholds on appenders to filter out messages below a certain level of importance but if you want to send (say) INFO messages to a particular appender but not also WARN, ERROR and FATAL, then the only way to achieve that is by applying a filter.

The properties configuration mechanism doesn't support filters, so you will have to switch to the log4j.xml configuration style. The following configuration example is taken from the Log4J wiki:

    <appender name="info-out" class="org.apache.log4j.FileAppender"> 
            <param name="File" value="info.log"/> 
            <layout class="org.apache.log4j.PatternLayout"> 
                    <param name="ConversionPattern" value="%m%n"/> 
            </layout> 
            <filter class="org.apache.log4j.varia.LevelMatchFilter">
                    <param name="LevelToMatch" value="info" />
                    <param name="AcceptOnMatch" value="true"/>
            </filter>
            <filter class="org.apache.log4j.varia.DenyAllFilter" />
    </appender> 
Ian Roberts
  • 120,891
  • 16
  • 170
  • 183
  • @IanRoberts «*properties configuration mechanism doesn't support filters*»: that statement is not exactly true. – Paul Vargas Jan 22 '15 at 20:39
  • @PaulVargas oh, ok, I was going by the documentation page I linked to which states that PropertyConfigurator doesn't do filters, I wasn't aware that information was out of date. – Ian Roberts Jan 22 '15 at 21:33