20

I need this ability to send push notifications for an action in a mobile app but wait for the user to undo the action until say 10 seconds.

Is it possible to delay the processing of a message published in a topic by 10 seconds ? And then (sometimes, if user does undo) delete the message before 10 seconds, if it doesn't need to be processed ?

iPhoney
  • 771
  • 1
  • 10
  • 14
  • Do you mean that you want to delay the publishing of a message and possibly not publish it at all or do you mean that upon a subscription receiving a message, you send a notification and potentially want to do something different with the message the subscription received based on how someone reacts to the notification? – Kamal Aboul-Hosn Oct 05 '16 at 20:07
  • @Kamal, I mean the latter. Publish message -> Delay delivery to subscribers by 10 seconds -> In these 10 seconds, I want to be able to delete the message from the queue (if user presses undo, typically within 10 seconds) -> If I don't delete, message is delivered to subscribers. I can build delayed delivery manually through a local queue mechanism but I was wondering if google-pubsub lets do that. – iPhoney Oct 07 '16 at 10:56
  • Turns out AWS SQS has this concept of delay queues: http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-delay-queues.html – iPhoney Oct 29 '16 at 07:35
  • @iPhoney, we're talking about a pubsub rather than a queue. GCP has that same functionality in it's TaskQueue service but it's not for M:N message system: https://cloud.google.com/appengine/docs/standard/python/taskqueue/?csw=1 – Gabriel Kohen Mar 22 '17 at 15:19

3 Answers3

26

Depends on if you write the subscribers as well or not:

You have control over your subscriber's code:

  1. In your PubSub messages add a timestamp for when you want that message to be processed.
  2. In your clients (subscribers), have logic to acknowledge the message only if the timestamp to process the message is reached.
  3. PubSub will retry delivering the message until it's acknowledged (or 10 days)

If you don't have control over your subscriber you can have a my-topic and my-delayed-topic. Folks can publish to the former topic and that topic will have only one subscriber which you will implement:

  1. Publish message as before to my-topic.
  2. You will have a subscriber for that topic that can do the same throttling as shown above.
  3. If the time for that message has reached your handler will publish/relay that message to my-delayed-topic.

You can also implement the logic above with task-queue+pubsub-topic instead of pubsub-topic+pubsub-topic.

Alexey Zimarev
  • 17,944
  • 2
  • 55
  • 83
Gabriel Kohen
  • 4,166
  • 4
  • 31
  • 46
  • 1
    thanks Gabriel, think I understand albeit vaguely. I haven't used pubsub yet, how frequently will be the retries if I delay ack ? The other part is can I delete a message using a handle / identifier if it is not cleared before the threshold time. – iPhoney Mar 28 '17 at 06:36
  • It will depend if it's a pull or a push topic. In general it will keep retrying up to 7 days. The interval of retry to quote: "Any of the following HTTP status codes is interpreted as success by the Pub/Sub system: 200, 201, 204, or 102. If your service returns any other code, Google Cloud Pub/Sub retries for up to 7 days, using an exponential backoff algorithm (capped at 10 seconds)". By default the ack deadline is 10 seconds but you can change it in your code. See this section: https://cloud.google.com/pubsub/docs/subscriber#ack_deadline – Gabriel Kohen Mar 29 '17 at 12:16
  • 1
    Thanks again for sharing how this could be made to work with pubsub. My 2 cents as the platform matures it may be useful to either exposes Queues as a separate product with a feature for delaying delivery or building the same in PubSub. – iPhoney Apr 01 '17 at 06:39
  • 1
    [Google Cloud Tasks](https://cloud.google.com/tasks/) already exists. But they do not have pull queues, just the push ones. – xmedeko May 06 '19 at 13:20
  • Dropping a Pub/Sub message without acking or nacking it is apparently not supported by the Java client and requires some workarounds https://github.com/googleapis/java-pubsub/issues/20 – David Xia May 28 '20 at 02:24
  • For future googlers: Google Pub/Sub bills for message throughput, this idea might send a lot of messages that will be nack'ed, those will incur a cost however. This might become expensive. More importantly: PubSub supports retry policies to handle op's issue: https://cloud.google.com/pubsub/docs/handling-failures – Kaalaamaazoo Jun 13 '22 at 09:32
3

If architecturally possible at all, you could use Cloud Tasks. This API has the following features that might suit your usecase:

  • You can schedule the delivery of the message (task)
  • You can delete the tasks from the queue (before they are executed)

Assuming that your client has a storage for some task Ids:

  1. Create a task with schedule_time set to 10s in the future.
  2. Store the task name in memory (you can either assign a name to the task at creation time, or use the automatically generated ID returned from the create response).
  3. If user undid the job, then call DeleteTask.
Ehsan
  • 92
  • 5
2

Just wanted to share that I noticed Pub/Sub supports retry policies 1 that are GA as of 2020-06-16 2.

If the acknowledgement deadline expires or a subscriber responds with a negative acknowledgement, Pub/Sub can send the message again using exponential backoff.

If the retry policy isn't set, Pub/Sub resends the message as soon as the acknowledgement deadline expires or a subscriber responds with a negative acknowledgement.

If the maximum backoff duration is set, the default minimum backoff duration is 10 seconds. If the minimum backoff duration is set, the default maximum backoff duration is 600 seconds.

The longest backoff duration that you can specify is 600 seconds.

David Xia
  • 5,075
  • 7
  • 35
  • 52