0

I have a number of backend processes (java applications) which run 24/7. To monitor these backends (i.e. to check if a process is not responding and notify via SMS/EMAIL) I have written another application.

The old backends now log heartbeat at regular time interval and this new applications checks if they are doing it regularly and notifies if necessary.

Now, We have two options

  • either run it as a scheduled task, which will run after every (let say) 15 min and stop after doing its job or
  • Run it as another backend process with 15 min sleep time.

The issue we can foresee right now is that what if this monitor application goes into non-responding state? So, my question is Is there any difference between both the cases or both are same? What option would suit my case more?

Please note this is a specific case and is not same as this or this

Environment: Java, hosted on LINUX server

Community
  • 1
  • 1
Umer Hayat
  • 1,993
  • 5
  • 31
  • 58

3 Answers3

1

By scheduled task, do you mean triggered by the system scheduler, or as a scheduled thread in the existing backend processes?

To capture unexpected termination or unresponsive states you would be best running a separate process rather than a thread. However, a scheduled thread would give you closer interaction with the owning process with less IPC overhead.

I would implement both. Maintain a record of the local state in each backend process, with a scheduled task in each process triggering a thread to update the current state of that node. This update could be fairly frequent, since it will be less expensive than communicating with a separate process.

Use your separate "monitoring app" process to routinely gather the information about all the backend processes. This should occur less frequently - whether the process is running all the time, or scheduled by a cron job is immaterial since the state is held in each backend process. If one of the backends become unresponsive, this monitoring app will be able to determine the lack of response and perform some meaningful probes to determine what the problem is. It will be this component that will then notify your SMS/Email utility to send a report.

seanhodges
  • 17,426
  • 15
  • 71
  • 93
  • The monitoring process will run as a separate process in both the cases. In 1st case it would be triggered by the system scheduler after every x min. In 2nd case in would be separate process running 24/7 with x min sleep time. – Umer Hayat Apr 16 '12 at 10:35
  • 1
    OK, as I explain in my answer the execution method of the monitoring process is academic as long as it holds no state. Personally I would probably make it a short-lived task triggered by a cron job, just to ensure I have one less long-running executable to worry about (monitoring apps can crash too). However, the complete lack of reports should be warning enough that something is wrong there. – seanhodges Apr 16 '12 at 10:42
0

I would go for a backend process as it can maintain state have a look at the quartz scheduler from terracotta http://terracotta.org/products/quartz-scheduler

It will be resilient to transient conditions and you only need provide a simple wrap so the monitor app should be robust providing you get the threading stuff right in the quartz.properties file.

Paddy Carroll
  • 528
  • 3
  • 20
  • Sorry I don't get how this is applicable in my case. These backends are part of application which hosted on third party servers. – Umer Hayat Apr 16 '12 at 09:57
  • you have already written the application, let quartz manage the scheduled invocation of the monitoring classes. – Paddy Carroll Apr 16 '12 at 10:09
  • 1
    using cron to invoke stuff can give you flexibility problems - lots of mail when planned maintenance kicks in – Paddy Carroll Apr 16 '12 at 10:16
  • We have an independent(SMS/EMAIL) notification already in place. We just want a process who can tell the notifier when to notify. We want to make it simple and less costly. Anyway, thanks a lot for your suggestion :) – Umer Hayat Apr 16 '12 at 10:20
0

You can use nagios core as core and Naptor to monitoring your application. Its easy to setup and embed with your application development.

You can check at this link: https://github.com/agunghakase/Naptor/tree/ver1.0.0

Agung Hakase
  • 1