How to automatically get email alerts on errors in Google app engine

Question

I maintain a lot of Google App Engine apps and it is very difficult to constantly keep a track of all my servers if they are running smoothly.

It is very embarrassing to wake up only after user complains about a system crash.

Is it possible to constantly get updated about my server health?

Can i get a mail or something in case their is a error on my google app engine application?

Can't wait for Google to take up this requirement

Answer 1

Now it is possible with Stackdriver (acquired by Google and integrated with Google Cloud Platform/App Engine).

1. Go to https://console.cloud.google.com/errors.
2. Click Turn on notifications

Answer 2

You can schedule a cron job which can do this for you

First create a entry in cron.yaml

- description: Houston we have a problem Finder
  url: /errorfinder/
  schedule: every 3 hours
  timezone: Asia/Kolkata

Then create an entry in app.yaml

handlers:
- url: /errorfinder/
  script: errorfinder.app
  secure: always 

Now keep errorfinder.py with the following content

import base64
import datetime
import logging
import time
import urllib
import webapp2
from google.appengine.api.logservice import logservice
from google.appengine.api import mail

class MainHandler(webapp2.RequestHandler):
    def get(self):    
      # Set up end time for our query.
      end_time = time.time()
      start_time = end_time  - 10800 # 3 hours before now . same as cronjob interval
      html = ''
      report_needed = False
      # Iterate through all the RequestLog objects, displaying some fields and
      # iterate through all AppLogs beloging to each RequestLog count times
      for req_log in logservice.fetch(start_time=start_time, end_time=end_time, minimum_log_level=logservice.LOG_LEVEL_WARNING, include_app_logs=True):
            report_needed = True
            html = html + '<br /> REQUEST LOG <br />'
            html = html + 'IP: %s <br /> Method: %s <br /> Resource: %s <br />' % (req_log.ip, req_log.method, req_log.resource)
            html = html + 'Date: %s<br />' % datetime.datetime.fromtimestamp(req_log.end_time).strftime('%D %T UTC')

            for app_log in req_log.app_logs:
                html = html + '<br />&emsp; APP LOG<br />'
                html = html + '&emsp; Date: %s<br />' % datetime.datetime.fromtimestamp(app_log.time).strftime('%D %T UTC')
                html = html + '&emsp; Message: <b>%s</b><br />' % app_log.message
                html = html + '<br /><br /><br /><br />'
      if(report_needed):
           mail.send_mail(sender="Beagle Bot <bot@urdomain.com>",
                to='lazyadmins@urdomain.com',
                subject='Houston we have a problem ..',
                body=html,
                html=html,
                reply_to='support@urdomain.com')
      self.response.out.write(html)

app = webapp2.WSGIApplication([('/errorfinder/', MainHandler)], debug=True)

You can make the error search close to real time by reducing the cron job interval

Answer 3

There is no automatic way to send alarm mails in case of errors.

You will need to make your own solution: create a cron job that parses logs and sends email about errors. Possibly aggregate similar errors together. Instead of emails you might consider automatically creating an error report in your issue tracking system.

Answer 4

For me, this link did the trick: https://console.cloud.google.com/user-preferences/communication?project=voilanorbert-1272&utm_source=cloud-notification&utm_medium=email&utm_content=unsubscribe-new-error

To access it, click on the three dot near your avatar on the top right, and select "Preferences".

Answer 5

You can configure your app to send you an email when exceptions are raised. See here for more details: Catch-All global exception handler in App Engine for Python