9

I have a Cosmos DB trigger for an Azure function. I want to flatten and write some data from the incoming Document(s) to an (Azure) SQL Server.

What is a way to guarantee at least once delivery?

I looked at https://hackernoon.com/reliable-event-processing-in-azure-functions-37054dc2d0fc which gives some options in the case of an Azure Function triggered by an Event Hub event, but I am not sure if the same applies for the CosmosDB changefeed that causes the trigger to fire.

On the Cosmos DB Change Feed site https://learn.microsoft.com/en-us/azure/cosmos-db/change-feed it states:

Each change to a document appears exactly once in the change feed, and clients manage their checkpointing logic. The change feed processor library provides automatic checkpointing and "at least once" semantics.

Does that mean that it implements the same (or something similar to) the checkpoint system from Event Hub?

Does the circuit breaker pattern work the same way if applied to this flow of a CosmosDB trigger to an Azure Function as detailed at the end of https://hackernoon.com/reliable-event-processing-in-azure-functions-37054dc2d0fc ?

kyarbles
  • 169
  • 1
  • 9
  • both azure storage queue and azure service bus queues gurantee at-least-once delivery, so post a message on any one of them and process it asynchronously using that – harishr May 08 '18 at 16:44
  • With this option how would an unsuccessful delivery in an Azure Function be communicated back to Service Bus? – kyarbles May 08 '18 at 17:12
  • you have to mark message as success or failed for unsuccessful delivery to be communicated, but answer from Mikhail look better than this option – harishr May 09 '18 at 04:30

1 Answers1

5

Azure Functions Cosmos DB trigger is based on Change Feed processor library. You will get at-least-once out of the box.

Mikhail Shilkov
  • 34,128
  • 3
  • 68
  • 107
  • What happens if there is a major "non-hiccup" type of problem and in the middle of 200 messages on message 105 the function is stopped and then restarted once an issue is fixed, will it process the entire backlog in order starting at 105? – kyarbles May 08 '18 at 17:10
  • @kyarbles If those 200 were part of one batch, then checkpoint won't be saved, so next time it will get the whole batch of 200 again – Mikhail Shilkov May 08 '18 at 17:16
  • thank you! Is there some documentation I can refer to for this? The Change Feed processor library one I couldn't see what you just mentioned. – kyarbles May 08 '18 at 19:05
  • 1
    @kyarbles [Azure Functions reliable event processing](https://learn.microsoft.com/en-us/azure/azure-functions/functions-reliable-event-processing) – giokoguashvili Dec 17 '20 at 09:20
  • If the runtime fails, it will move forward. Sadly. https://learn.microsoft.com/en-us/azure/cosmos-db/nosql/troubleshoot-changefeed-functions#some-changes-are-missing-in-my-trigger – Narvalex Dec 09 '22 at 21:14