Will PubSub forward message to dead letter topic after delivery_attempt exceeds max_delivery_attempts

Question

My subscriber looks like this:

from google.cloud import pubsub_v1
from google.cloud.pubsub_v1.types import DeadLetterPolicy

dead_letter_policy = DeadLetterPolicy(
    dead_letter_topic='dead_letter_topic',
    max_delivery_attempts=5,
)

topic_path = subscriber.topic_path(PROJECT, TOPIC)
subscriber.create_subscription(sub_path, topic_path, dead_letter_policy=dead_letter_policy)

subscriber = pubsub_v1.SubscriberClient()
subscription_path  = subscriber.subscription_path(PROJECT, SUBSCRIPTION)

def callback(message):
    print("Received message: {}".format(message))
    print('Attempted:', message.delivery_attempt, 'times')
    data = message.data.decode('utf-8')
    data_d = json.loads(data)
    if data_d["name"] == "some_file.json":
        message.nack()
    else:
        message.ack()

The received message looks like this:

Received message: Message {
  data: b'{\n  "kind": "storage#object",\n  "id": "...'
  attributes: {
    "bucketId": "sample_bucket",
    ...
  }
}
Attempted: 12 times

Clearly it attempted more than 5 times, why I can still pull this message from PubSub topic? Here is the subscription info:

ackDeadlineSeconds: 10
deadLetterPolicy:
  deadLetterTopic: projects/someproject/topics/dead_letter
  maxDeliveryAttempts: 5
expirationPolicy:
  ttl: 2678400s
messageRetentionDuration: 604800s
name: projects/someproject/subscriptions/new_sub
pushConfig: {}
topic: projects/someproject/topics/pubsub_sample

You can find the updated documentation to use Dead Letter Queues [here](https://cloud.google.com/pubsub/docs/dead-letter-topics). — Mahesh Gattani, Apr 23 '20 at 16:22

score 3 · Accepted Answer · answered Feb 08 '20 at 02:00

Typically, this happens if you haven't given Pub/Sub permission to publish to your dead letter topic or subscribe to your subscription. You need to ensure you have run the following:

PUBSUB_SERVICE_ACCOUNT="service-${PROJECT_NUMBER}@gcp-sa-pubsub.iam.gserviceaccount.com"

gcloud pubsub topics add-iam-policy-binding <dead letter topic> \
 --member="serviceAccount:${PUBSUB_SERVICE_ACCOUNT}"\
 --role='roles/pubsub.publisher'

gcloud pubsub subscriptions add-iam-policy-binding <subscription with dead letter queue> \
 --member="serviceAccount:${PUBSUB_SERVICE_ACCOUNT}"\
 --role='roles/pubsub.subscriber'

If writing to the dead letter queue topic fails, then Cloud Pub/Sub will continue to deliver the message to your subscriber.

Can we do this policy bindings from python code? I am creating topics and subscriptions at run time and would like to attach dead letter policy to these subscriptions, but can not find docs showing how to change these policy bindings at run time — Rohit Shisode, Dec 28 '21 at 05:31

Will PubSub forward message to dead letter topic after delivery_attempt exceeds max_delivery_attempts

1 Answers1