Whats the best way to notify admin about new exceptions of the Java application?

Question

My question is that whats the best way to keep track of the exceptions for the administrator of the application. (Notify administrator of the thrown exceptions for maintenance purposes).

To users of the system, I believe should catch the exceptions and show an appropriate error message. To admin of the system, I suppose, best method is to have a messaging system to send details of each exception as a message to the receiver. Once receiver received a new error message persists it in the database or pings the admin an email with details of the exception.

try{
  ....
}
catch(Exception e){
   //what to do here? how to notify admin?
}

Answer 1

I'd suggest using log4j, configured with an SMTPAppender listening to fatal logs. Then, just log a fatal level message (containing any useful information you can get) for any unhandled exception reaching your global try/catch block.

See also : What is the proper way to configure SMTPAppender in log4j?

Answer 2

Enterprise solution:

Use SL4J and save all messages to your logs.

Use MDC to add tags to your log messages. Have these tags describe who should be notified and the nature of the error:

2014-05-24 [SystemCAD][NOTIFY=ADMIN], [ACCOUNTID=123], [SEVERITY=SEVERE], [MESSAGE="Cannot contact Google.com"]  
2014-05-24 [SystemCAD][NOTIFY=USER], [ACCOUNTID=123], [SEVERITY=SEVERE], [MESSAGE="Could not save document to Google. Support has been notified."]  

Get Splunk or some product similar to index all your logs for easy searching and to create events which can be used to notify your admins. Use PagerDutty to notify your admins and create escalation, avoid duplicates, create triggers, etc.

Answer 3

First, do not try to solve the notification problem in the application itself.

The recommended approach is to catch the exception at an appropriate point in the application and generate a log event that captures the details (including the exception) of the failure. The primary logging should be done using a standard logging system. There are a number of viable options (e.g. java.util.logging, log4j, logback, log4j2, slf4j), each with pro's and con's, but the most important thing is to not attempt to "roll your own".

That's the easy part.

The hard part is figuring out how to get the notification from the logging system to the admin in a way that is appropriate. There are many things that need to be considered:

• The admin does not be woken up at 2am by a page reporting an over-temperature in the office water cooler.

• The admin does not want 50 SMS message all reporting the same problem. The system needs to be able to filter out duplicates.

• The admin needs to be able to tell the system to "shut up" about a certain problem / issue.

• The system needs to recognize that certain events are more important than others, and that business hours versus after hours affects prioritization.

• What is the most appropriate way to notify the admin? Email? SMS? Pager?

• Escalation - what if the primary (on-call) admin does not respond to the notification?

• The system also needs to be integrated with other monitoring; e.g. checking service availability, network connectivity, file system levels, CPU / load average measures, checking that important events DO happen.

• All of this needs to be configurable, independent of the application that generated the event in the first place.

• Ideally, you need integration with operational issue tracking system ... to help the admin relate the event to previous problems, etc.

This is a really big problem space. Fortunately, there are products out there that do this kind of thing. Too many to list here.

(IMO, it doesn't make sense to recommend a solution for you. We don't know your organization's requirements. This is something that needs to be sorted out in conjunction with operational staff & management.)

Answer 4

I have done notification on exception in my application using spring AOP.

For example

@Aspect
public class ExceptionAspect {

   @AfterThrowing(
      pointcut = "execution(* com.suren.customer.bo.CustomerBo.addCustomerThrowException(..))",
      throwing= "error")
    public void logAfterThrowing(JoinPoint joinPoint, Throwable error) {
    // Notify admin in email
    sendEmail(joinPoint,error);

    }
}

Common AspectJ annotations :

@Before – Run before the method execution
@After – Run after the method returned a result
@AfterReturning – Run after the method returned a result, intercept the returned result as well.
@AfterThrowing – Run after the method throws an exception
@Around – Run around the method execution, combine all three advices above.

Answer 5

When you design an application you need to consider two types of exceptions

• User defined business exception
• Unexpected system exception

User defined exceptions

User defined exceptions are used to pass negative conditions from one layer to another (service to web). For example in a banking application, if there is no balance in an account and if you try to withdraw money, WithdrawService might throw NoBalanceException. The web layer would catch this exception and display appropriate message to the user.

These type of exceptions are of no interest to the administrators and no alert is required. You may simply log it as info.

Unexpected system exception

Unexpected system exceptions are exceptions like database connectivity or JMS conncetivity or NullPointException or invalid message received from external system. Basically any unexpected (non-business) exceptions are classified as system exceptions.

According to Joshua Bloch in Effective Java, it is advisable not to catch system exception as you might do more harm than good. Instead allow it to propagate to the highest level (web layer).

In my applications, I provide a global exception handler (supported by Spring / Struts 2) on the web layer and send a detailed email to the ops team including the exception stack trace and redirect the request to a standard error page which says something like "Unexpected internal error occurred. Please try again".

Using this option is more secured as it will not expose the ugly exception stack trace to the user in any situation.

Struts2 reference: http://struts.apache.org/release/2.3.x/docs/exception-handling.html

Answer 6

You should use a logging facility to log every exception in a file system so if Admin want they can view it through file-system.

ErrorUtil

public class ErrorLogUtil {

    public static File createErrorFile(String fileName, String productName,
            String regionName) {
        File fileErrorLogs = new File("Error Logs");
        if (!fileErrorLogs.isDirectory()) {
            fileErrorLogs.mkdir();
        }
        File fileProductName = new File(fileErrorLogs, productName);
        if (!fileProductName.isDirectory()) {
            fileProductName.mkdir();
        }

        File fileDate = null;

        if (regionName != null && regionName.trim().length() != 0) {
            File fileRegionName = new File(fileProductName, regionName);
            if (!fileRegionName.isDirectory()) {
                fileRegionName.mkdir();
            }

            fileDate = new File(fileRegionName, new SimpleDateFormat(
                    "dd-MM-yyyy").format(new Date()));
            if (!fileDate.isDirectory()) {
                fileDate.mkdir();
            }
        } else {
            fileDate = new File(fileProductName, new SimpleDateFormat(
                    "dd-MM-yyyy").format(new Date()));
            if (!fileDate.isDirectory()) {
                fileDate.mkdir();
            }
        }

        File errorFile = new File(fileDate, fileName + "-errors.txt");
        try {
            if (!errorFile.exists()) {
                errorFile.createNewFile();
                System.out.println("New Error File created=>"+errorFile.getAbsolutePath());
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
        return errorFile;
    }

    public static void writeError(File errorFile, String error) {
        try {
            FileOutputStream fileOutputStream = new FileOutputStream(errorFile,
                    true);
            DataOutputStream out = new DataOutputStream(fileOutputStream);
            BufferedWriter bufferedWriter = new BufferedWriter(
                    new OutputStreamWriter(out));
            bufferedWriter.append((new Date())+" - "+error);
            bufferedWriter.newLine();
            bufferedWriter.flush();
            bufferedWriter.close();
            fileOutputStream.flush();
            fileOutputStream.close();
            out.flush();
            out.close();


        } catch (IOException e) {
            e.printStackTrace();
        }
    }

    public static void printStackTrace(File errorFile, String message, Throwable error) {
        try {
            FileOutputStream fileOutputStream = new FileOutputStream(errorFile,
                    true);
            DataOutputStream out = new DataOutputStream(fileOutputStream);
            PrintWriter bufferedWriter = new PrintWriter(
                    new BufferedWriter(new OutputStreamWriter(out)));

            bufferedWriter.println(new Date() + " : "+ message);        

            error.printStackTrace(bufferedWriter);

            bufferedWriter.println();
            bufferedWriter.close();

        } catch (IOException e) {
            e.printStackTrace();
        }
    }

}

Sending mail will not be good because it may fill Admin's mail box but if you really need this you can create a MailUtil and send emails to the user or keep it in a log.

MailUtil

public class MailUtil {
    public static void sendEmail(String messageString, String subject, Properties props) {

        try {
            Session mailSession = null;
            final String userName = props.getProperty("mail.from");
            final String password = props.getProperty("mail.from.password");
            mailSession = Session.getInstance(props, new javax.mail.Authenticator() {
                protected PasswordAuthentication getPasswordAuthentication() {
                    return new PasswordAuthentication(userName, password);
                }
            });

            Transport transport = mailSession.getTransport();

            MimeMessage message = new MimeMessage(mailSession);

            message.setSubject(subject);
            message.setFrom(new InternetAddress(props.getProperty("mail.from")));
            String[] to = props.getProperty("mail.to").split(",");
            for (String email : to) {

                message.addRecipient(Message.RecipientType.TO, new InternetAddress(email));
            }

            String body = messageString;
            message.setContent(body, "text/html");
            transport.connect();

            transport.sendMessage(message, message.getRecipients(Message.RecipientType.TO));
            transport.close();
        } catch (Exception exception) {
            exception.printStackTrace();
        }
    }

    public static void sendEmail(String subject, String messageString) {
        try {
            Session mailSession = null;
            Properties props=new Properties();
            FileInputStream fileInputStream = new FileInputStream(new File("mail-config.properties"));
            props.load(fileInputStream);
            fileInputStream.close();

            final String fromUsername = props.getProperty("mail.from");
            final String fromPassword = props.getProperty("mail.from.password");

            mailSession = Session.getInstance(props, new javax.mail.Authenticator() {
                protected PasswordAuthentication getPasswordAuthentication() {
                    return new PasswordAuthentication(fromUsername, fromPassword);
                }
            });

            Transport transport = mailSession.getTransport();

            MimeMessage message = new MimeMessage(mailSession);

            message.setSubject(subject);
            message.setFrom(new InternetAddress(fromUsername));
            String[] to = props.getProperty("mail.to").split(",");
            for (String email : to) {
                message.addRecipient(Message.RecipientType.TO, new InternetAddress(email));
            }

            String body = messageString;
            message.setContent(body, "text/html");
            transport.connect();

            transport.sendMessage(message, message.getRecipients(Message.RecipientType.TO));
            transport.close();
        } catch (Exception exception) {
            exception.printStackTrace();
        }
    }
}

You should use a property to manage if mail is required or not so in future you can stop mails by just changing the property file.

Answer 7

Consider using standard logging (like log4j) and using appender suited for you - either SMTP mentioned before, or custom one. There exists solutions called logging servers - they provide high flexibility in terms of notifications, filtering, storing, processing etc. Good place to start reading and investigating are Scribe and Flume. A great discussion on this subject may be found here.

There are also some cloud solutions available, from automated ones like Sentry through LogDigger (your own installation) to more low-level setups like Amazon SQS.

Answer 8

you can create exception log table. There, Write a code to insert exception with "Pending" status into database whatever exception raised in application. Create a cron job (linux) or quartz scheduler that will fired in certain period and send mail of "pending" status exception with predefined format to admin user. Update database entry with "sent" status so it will not send again.

In code, To save exception create super class,i.e.

class UserDao extends CommonDao
{

  try
   {

   }catch(Exception e)
   {
      saveException(e);
   }
}

class CommonDao
{

  public void saveException(Exception e)
  {
    //write code to insert data into database
  }
}

Answer 9

For me is not a good idea to put that behavior directly in the code of the application. It is clear that simply call to a function that sends an email in the catch clause it is easy, fast and direct. If you haven't so much time go for it.

But then you will realize that will produce some not expected collateral effects you will need to

• Control the performance on exception parsing
• Control what exceptions are interesting to notify and what not
• Control not send lot of emails because a bug in the application producing exceptions constantly.

For that I prefer to use http://logstash.net/ That allows to put all your logs in a common noSQL database, and then you can use logstash to make dashboards or even create your own applications to send well designed reports about specific events. It require a bit more work at the beginning but after that I'm sure you will have more control about what is important to see in the logs and what not.