How to keep data consistent when sync documents with cloud firestore triggers?

Question

I have a document MAIN which holds an array of objects. On every change of this array i want to update related documents. So in the example below If I add something to the targets of MAIN i want to grab all documents where the field "parent" holds a reference to MAIN and then update their targets accordingly. I wanted to do this with cloud functions so the client does not have to care about updating all related documents himself. But as stated in the docs cloud-funcion triggers do not guarantee order. So if f.e. a user adds a new object to the targets of MAIN and then removes it, cloud trigger would maybe receive the remove event before the add event and thus RELATED documents would be left with inconsistent data. As stated by Doug Stevenson in this stackoverflow post this could happen, even when using transactions. Am I right so far?

  const MAIN = {
    ID:"MAIN"
    targets:[{name: "House"}, {name:"Car"}]
  }

  const RELATED_1 = {
    parent: "MAIN",
    targets:[{name: "House"}, {name:"Car"}]
  }

  const RELATED_2 = {
    parent: "MAIN",
    targets:[{name: "House"}, {name:"Car"}]
  }

If yes, I was thinking about adding a servertimestamp to object MAIN whenever I modify the Document. I would use this timestamp to only update RELATED Documents if their timestamp is smaller then the one of the parent. If yes I update the array and set the timestamp of the parent.

  const MAIN = {
    ID:"MAIN",
    targets:[{name: "House"}, {name:"Car"}],
    modifiedAt: 11.04.2022 10:25:33:233
  }

  const RELATED_1 = {
    parent: "MAIN",
    lastSync: 11.04.2022 10:25:33:233,
    targets:[{name: "House"}, {name:"Car"}]
  }
  
  
  const RELATED_1 = {
    parent: "MAIN",
    lastSync: 11.04.2022 10:25:33:233,
    targets:[{name: "House"}, {name:"Car"}]
  }

Would that work? Or how could one sync denormalized data with cloud functions and keep data consistent? Is this even possible?

Those links explain on how to update an array and how to update multiple documents with a batch. What I am looking for is a way to asynchronously update denormalized data while guarantee consistency. Doing it with cloud-functions seems to not really work, because it does not guarantee that the events come in order. So if a client modifies a field two times, maybe the first modification is handled second and thus will be the new value of the synced documents instead of the second modifications value. It's not really related to arrays, it's just my example, maybe not the best. — KindaOk, Apr 12 '22 at 13:21
you can execute multiple write operations as a single batch using [batched writes](https://firebase.google.com/docs/firestore/manage-data/transactions), also check stackoverflow [link1](https://stackoverflow.com/a/30699277/18265638) ,[link2](https://stackoverflow.com/questions/54254581/managing-denormalized-duplicated-data-in-cloud-firestore) &[link3](https://stackoverflow.com/questions/47499820/multi-path-update-with-firestore),[data consistency with multiple updates](https://www.youtube.com/watch?v=i1n9Kw3AORw) — Sathi Aiswarya, Apr 13 '22 at 08:05
I know that it would be possible using batched-writes. But this is not async and thus may takes a lot of time until the actual change is visible for the user. The multi-path updates I guess is something related to realtime-database and not firestore. — KindaOk, Apr 13 '22 at 12:42

Sathi Aiswarya · Answer 1 · 2022-04-15T05:58:34.023

Using the Cloud Firestore client libraries, you can group multiple operations into a single transaction. Transactions are useful when you want to update a field's value based on its current value, or the value of some other field.

A transaction consists of any number of get() operations followed by any number of write operations such as set(), update(), or delete(). In the case of a concurrent edit, Cloud Firestore runs the entire transaction again. For example, if a transaction reads documents and another client modifies any of those documents, Cloud Firestore retries the transaction. This feature ensures that the transaction runs on up-to-date and consistent data.

Transactions are a way to always ensure a write occurs with the latest information available on the server. Transactions never partially apply writes & all writes execute at the end of a successful transaction.

For a transaction to succeed, the documents retrieved by its read operations must remain unmodified by operations outside the transaction. If another operation attempts to change one of those documents, that operations enters a state of data contention with the transaction.

A transaction works differently than a regular update. It goes like this:

Firestore runs the transaction.
You get the document ready to update whatever property you want to update. Firestore checks if the document has changed. If not, you’re good, and your update goes through.
If the document has changed, let’s say a new update happened before yours did, then Firestore gets you the new version of the document and repeats the process until it finally updates the document.

The following example from firebase documentation shows how to create and run a transaction:

// Initialize document
const cityRef = db.collection('cities').doc('SF');
await cityRef.set({
  name: 'San Francisco',
  state: 'CA',
  country: 'USA',
  capital: false,
  population: 860000
});

try {
  await db.runTransaction(async (t) => {
    const doc = await t.get(cityRef);

    // Add one person to the city population.
    // Note: this could be done without a transaction
    //       by updating the population using FieldValue.increment()
    const newPopulation = doc.data().population + 1;
    t.update(cityRef, {population: newPopulation});
  });

  console.log('Transaction success!');
} catch (e) {
  console.log('Transaction failure:', e);
}

For more information on the above can refer to how do transactions work ,updating data and fastest way to perform bulk data creation

How to keep data consistent when sync documents with cloud firestore triggers?

1 Answers1