9

I have integrated pagerduty with AWS cloudwatch and I am trying to publish a message manually to a SNS Topic that is subscribed by pagerduty and email. But I am not able to get incidents in pagerduty. However, cloudwatch alarms are triggering incidents in pagerduty using this same topic.

I referred some document for pagerduty message payload. But unable to make it work. My SNS message JSON is as follows,

{
 "default":"test message",
 "email":"test email message",
 "https":{
    "service_key":"XXXX",
    "event_type":"trigger",
    "description":"Example alert on host1.example.com"
  }
}

Its not triggering an incident in pagerduty. I am not sure what I am missing in the request body. I am receiving email messages properly from this same message body. Could someone point out the mistake?

Thanks in advance.

Sujai Sivasamy
  • 1,101
  • 3
  • 14
  • 32
  • I used email integration in pagerduty to achieve this rather than API integration. Its working fine that way and is also easy to implement. – Sujai Sivasamy Nov 12 '18 at 05:50
  • 2
    I asked the same question to PagerDuty support and they pointed me to https://v2.developer.pagerduty.com/docs/creating-an-integration-inline to see the message formats. If I figure this out I'll come back – brian d foy Jan 10 '19 at 14:03
  • There's now an API v2 with this: https://developer.pagerduty.com/docs/app-integration-development/events-integration/ – Efren Dec 09 '20 at 07:27
  • 1
    I was able to do it and documented it here https://blog.cetinich.net/content/2022/cloudwatch-pagerduty-sns/ – Brent Oct 16 '22 at 11:59

5 Answers5

4

To do so, you must choose the option Custom Event Transformer for the PagerDuty Integration. In the integration, you can write your own JavaScript code as follows:

var normalized_event = {
    event_type: PD.Trigger,
    description: "SNS Event",
    details: PD.inputRequest
};

PD.emitGenericEvents([normalized_event]);

To parse the received payload from SNS, you can use:

var rawBody = PD.inputRequest.rawBody;
var obj = JSON.parse(unescape(rawBody));

And treat obj to treat your event according to your SNS message.

filipebarretto
  • 1,842
  • 1
  • 13
  • 27
3

I believe PagerDuty's native AWS CloudWatch integration is opinionated. So a Custom SNS message won't trigger an incident.

But PagerDuty has an inbound integration type that allows you to create a script using JS (ES5) to parse any custom message sent to the this integration - which can then trigger an incident based on the logic of your script.

Docs on the Custom Event Transformer: https://v2.developer.pagerduty.com/docs/creating-an-integration-inline

Jay C
  • 103
  • 1
  • 5
3

I'm too late to answer this but still adding as @filipebarretto has suggested we need to use Custom Event Transformer for this type of integration.

Setup: ~ AWS Cloudwatch (RDS Metric) -> AWS SNS -> PagerDuty (CET)

I have successfully integrated AWS SNS to PagerDuty via Custom Event Transformer

var body = JSON.parse(PD.inputRequest.rawBody)
var message = body.NewStateReason

var normalized_event = {
      event_type: PD.Trigger,
      description: body.AlarmName,
      details: message
    };
PD.emitGenericEvents([normalized_event]);

The above code will send incident as AlarmName and details as NewStateReason.

Hussain K
  • 613
  • 8
  • 16
2

I tested with below sample events as SNS message, it works fine.

{
  "version": "0",
  "id": "bba1bcef-5268-9967-8628-9a6d09e042e9",
  "detail-type": "CloudWatch Alarm State Change",
  "source": "aws.cloudwatch",
  "account": "[Account ID]",
  "time": "2020-11-17T06:25:42Z",
  "region": "[region Id]",
  "resources": [
    "arn:aws:cloudwatch:[region Id]:[Account ID]:alarm:CPUUtilize"
  ],
  "detail": {
    "alarmName": "CPUUtilize",
    "state": {
      "value": "ALARM",
      "reason": "Threshold Crossed: 1 out of the last 1 datapoints [4.314689265544354 (17/11/20 06:20:00)] was less than the threshold (70.0) (minimum 1 datapoint for OK -> ALARM transition).",
      "reasonData": {
        "version": "1.0",
        "queryDate": "2020-11-17T06:25:42.491+0000",
        "startDate": "2020-11-17T06:20:00.000+0000",
        "statistic": "Average",
        "period": 300,
        "recentDatapoints": [
          4.314689
        ],
        "threshold": 70
      },
      "timestamp": "2020-11-17T06:25:42.493+0000"
    },
    "previousState": {
      "value": "OK",
      "reason": "Threshold Crossed: 1 out of the last 1 datapoints [4.484088172640544 (17/11/20 05:44:00)] was not greater than or equal to the threshold (70.0) (minimum 1 datapoint for ALARM -> OK transition).",
      "reasonData": {
        "version": "1.0",
        "queryDate": "2020-11-17T05:49:53.688+0000",
        "startDate": "2020-11-17T05:44:00.000+0000",
        "statistic": "Average",
        "period": 300,
        "recentDatapoints": [
          4.484088
        ],
        "threshold": 70
      },
      "timestamp": "2020-11-17T05:49:53.691+0000"
    },
    "configuration": {
      "description": "Alarm Notification in my local timezone",
      "metrics": [
        {
          "id": "16baea70-421b-0a6e-f6f1-bc913d2bf647",
          "metricStat": {
            "metric": {
              "namespace": "AWS/EC2",
              "name": "CPUUtilization",
              "dimensions": {
                "InstanceId": "i-0e448XXXXXXXXXXXX"
              }
            },
            "period": 300,
            "stat": "Average"
          },
          "returnData": true
        }
      ]
    }
  }
}

Took from https://aws.amazon.com/blogs/mt/customize-amazon-cloudwatch-alarm-notifications-to-your-local-time-zone-part-1/

Yon J
  • 61
  • 1
0

I am even later to the game here, but ...

How are you 'manually' sending the events? Did you check that the Policy on the SNS topic allows publishing of notifications from whichever service you are using to publish the events?

I had a similar issue with publishing notifications/events from AWS Backup. I had to add something like this to the Access Policy:

{
      "Sid": "My-statement-id",
      "Effect": "Allow",
      "Principal": {
        "Service": "backup.amazonaws.com"
      },
      "Action": "SNS:Publish",
      "Resource": "arn:aws:sns:region:account-id:myTopic"
}
schrom
  • 1,372
  • 1
  • 22
  • 36
Roy Long
  • 11
  • 1