Both "Delivery" and "Bounce" SNS Notifications after sending via AWS SES

Question

I have these really awkward situation, perhaps I'm misunderstanding the use of these notifications.

I've set up AWS SES to publish to a topic after sending emails out. I've set it to publish for Bounce, Complaint, and Delivery.

What I do is, when I receive an SNS notification on my web server, I will look up that message id in my database, and then change its status. For instance, if delivery notification arrives, I change the message status to "Delivered". If bounce notification arrives, I change the message status to "Bounced".

However, now I notice that many of these emails are sending me both notifications, and they are not always in a specific order. Sometimes one comes earlier than the other, and sometimes vice-versa.

So if "Bounce" arrives first, and then "Delivered", my status in my database for this message becomes "Delivered", which I thought is probably misleading.

First question How do I check the sequence of these notifications?

Second question I feel that I'm misunderstanding the "Delivery" notification. I've tried reading the AWS docs, but to be honest, they weren't the best docs on Earth. Can anyone give me a simple, clear explanation on this?

Third question Am I even handling these notifications correctly? Or is there a better way?

Your help is appreciated.

Thanks!

--

For your information, I've attached a sample, which consists of a set of 2 notifications. One bounce, and one delivery.

{
    "Type": "Notification",
    "MessageId": "2efb9ee6-6bd5-576d-b80d-d0f5e3f44f23",
    "TopicArn": "arn:aws:sns:us-east-1:#####:ses_beamstyle_com_hk",
    "Message": {
        "notificationType": "Delivery",
        "mail": {
            "timestamp": "2015-07-05T19:30:40.441Z",
            "source": "<no-reply@bs.com.hk>",
            "messageId": "xxxxx
            "destination": [
                "<ooto@simulator.amazonses.com>"
            ]
        },
        "delivery": {
            "timestamp": "2015-07-05T19:30:41.101Z",
            "processingTimeMillis": 660,
            "recipients": [
                "ooto@simulator.amazonses.com"
            ],
            "smtpResponse": "250 2.6.0 Message received",
            "reportingMTA": "a9-140.smtp-out.amazonses.com"
        }
    },
    "Timestamp": "2015-07-05T19:30:41.179Z",
    "SignatureVersion": "1",
    "Signature": "xxxxx",
    "SigningCertURL": "xxxxx",
    "UnsubscribeURL": "xxxxx"
}





{
    "Type": "Notification",
    "MessageId": "b6e8bb7d-4e73-52ae-b690-f56ec6527ce5",
    "TopicArn": "arn:aws:sns:us-east-1:#####:ses_beamstyle_com_hk",
    "Message": {
        "notificationType": "Bounce",
        "bounce": {
            "bounceSubType": "General",
            "bounceType": "Transient",
            "bouncedRecipients": [
                {
                    "emailAddress": "ooto@simulator.amazonses.com"
                }
            ],
            "timestamp": "2015-07-05T19:30:41.000Z",
            "feedbackId": "xxxxx"
        },
        "mail": {
            "timestamp": "2015-07-05T19:30:40.000Z",
            "messageId": "xxxxx",
            "destination": [
                "<ooto@simulator.amazonses.com>"
            ],
            "source": "<no-reply@bs.com.hk>"
        }
    },
    "Timestamp": "2015-07-05T19:30:41.315Z",
    "SignatureVersion": "1",
    "Signature": "xxxxx",
    "SigningCertURL": "xxxxx",
    "UnsubscribeURL": "xxxxx"
}

Answer 1

The nature of SMTP delivery, which is what SES uses for all outbound delivery (regardless of whether you use SMTP or the API) is such that both bounces and deliveries actually do sometimes occur on the same message.

To draw a parallel, imagine I ask you to telephone a company on my behalf, and leave a message for Jane Smith, even though (unknown to you) there's not an individual at that company by that name. One of two things will happen:

• before the call ends, you'll be told, "I'm sorry, there is no one here by that name," or,

• you'll leave a message, and end the call, because the person taking the message assumes there is someone by that name, or perhaps it's the name of a former employee that the call taker doesn't realize is no longer present.

In the first scenario, if I ask, "did you leave a message for Jane?" you'd say "no." But in the second scenario, you'd say "yes."

But ... after you end the call, in the second scenario, someone at the company will subsequently realize the message is addressed to an unknown person. If they are courteous, they may call you back to say, "we can't deliver your message, because there is no one here by that name."

Now, you call me and say, "I couldn't deliver that message." "Wait, what? You told me you already did."

But the reason why you didn't actually deliver that message, even though you said you did, is clear enough.

The same problem occurs on email. Arguably, the correct behavior for an incoming mail server is to immediately reject a message that won't be deliverable.

An unfortunately vast number of email providers do not validate the email address when it is presented at the beginning of an SMTP transaction. Instead, they accept all mail for the domains they should, and defer that deliverability check until later, so that it's impossible for them to actively reject an incoming message... which is what would be necessary for SES to avoid sending a "false alarm" delivery notification. By deferring the check, it becomes necessary for the recipient system to actually generate an email back to the originator (which turns out to be SES because of the way they format the messages). In the absence of a genuine bounce, SES first thinks the mail was delivered, and then thinks the mail bounced.

In most cases, an email that returns a bounce notification from SES has, indeed, bounced... regardless of any delivery notification you receive. Typically, delivery should come first, but it's possible to get them out of order -- though that should be the exception.

For ISP bounces, Amazon SES reports only hard bounces and soft bounces that will no longer be retried by Amazon SES. In these cases, your recipient did not receive your email message, and Amazon SES will not try to resend it.

http://docs.aws.amazon.com/ses/latest/DeveloperGuide/notifications.html

That is straightforward enough... But, of course, there's an annoying exception:

You are notified of out-of-the-office (OOTO) messages through the same method as bounces, although they don't count toward your bounce statistics.

Out of office responses, on the wire, look very much like bounces. Presumably, when SES can't determine a better type of bounce to report, based on the construction of the bounce message that's sent back after the destination has already -- apparently -- accepted your message... they use what you see in your example:

        "bounceSubType": "General",
        "bounceType": "Transient",

See http://docs.aws.amazon.com/ses/latest/DeveloperGuide/notification-contents.html#bounce-types for the possible combinations.

Your best approach is to probably store all the responses, in the event this turns out to be a less than accurate assessment, and consider the Transient/General bounce notification to be an out-of-office alert.

Other Transient subtypes may be interpreted as someone to whom you could successfully send to in the future, while Permanent subtypes are addresses you should avoid sending to, since the address is considered permanently undeliverable.

With the exception of out-of-office, bounces should (absent misconfiguration on the recipient side) be a reasonably reliable indicator to you that this particular message did not go through.