How to write more than 500 documents using batch in cloud firestore?

Question

I am trying to update more than 500 documents like 1000 to 2000 documents and i only know how to use batch for 500 documents i wanted to ask how can i update more than 500 documents using cloud firestore.here is how i am trying to update 500 documents. i am trying to update ms_timestamp for 1000 documents. can anyone tell me how can I do it using batch write

const batch = db.batch();
const campSnapshot = await db
  .collection("camp")
  .where("status", "in", ["PENDING", "CONFIRMED"])
  .get();
await db.collection("camping").doc(getISO8601Date()).set({
  trigger: campSnapshot.docs.length,
});
campSnapshot.forEach((docs) => {
  const object = docs.data();
  object.ms_timestamp = momentTz().tz("Asia/Kolkata").valueOf();

  batch.set(
    db.collection("camp").doc(docs.get("campId")),

    object,
    { merge: true }
  );
});
await Promise.all([batch.commit()]);

The 500 limit is a hard limit. You can simply perform multiple batches, to add more documents, but you can't exceed the limit. — Doug Stevenson, Jun 08 '20 at 15:50
@DougStevenson i want to use multiple batches to perform the task. — lucifer luci, Jun 09 '20 at 04:51

score 1 · Answer 1 · answered Jun 08 '20 at 10:56

Cloud Firestore imposes a limit of 500 documents when performing a Transaction or Batched Write, and you can not change this, but a workaround may just work.

I am not an expert in web dev, so I am sharing a suggestion based on my viewpoint as a mobile app developer.

Create a collection that store a counter on how many documents that are contained within a specific collection. I update the counter through Cloud Functions/other approach, when an event (be it created, updated, or deleted) is fired within that specific collection. The counter should be atomic and consistent, and you can leverage Cloud Firestore Transaction here.
Fetch the counter value before performing batched writes. Here, I will know how many data/objects/documents that need to be updated.
Create an offset with initial value is 0. The offset is used to mark the data. A batched writes can only be performed for up to 500 documents, so if I want to perform a batched write again on document/data at 501-1000, then the offset will be 500, and so on.
Call a method that perform batched writes recursively using the defined offset, until it fully equals the counter - 1.

I do not test this since I don't have enough time right now, but I think it'll work.

You can comment if you still not understand, I'll be glad to help further.

How to write more than 500 documents using batch in cloud firestore?

1 Answers1