How can I update a large number of documents in my Firestore database?

Question

Background

I'm a young developer still in my undergraduate program so I apologize in advance if I seem ignorant. I am, I'm learning and this solution is testing my abilities. It's fun though hopefully, I can get some help.

Goal

I need to update every document within one of my Firestore collections. Each document needs additional data from its corresponding user document.

For example: The document that needs to be updated contains "John" and "Amanda" 's uids. I need to add additional fields to their shared document that is each of their "FullName": "John Adams" and "Amanda Smith"

Before Update:

{
  "Document"
    "userA": "John",
    "userB": "Amanda"
}

After Update:

{
  "Document"
    "userA": "John",
    "userAFull": "John Adams",
    "userB": "Amanda",
    "userBFull": "Amanda Smith"
}

Approach

Since I have to update every document within the collection (Approx 20k documents) I decided to write a Cloud Function using chained batched writes to solve this problem so that all the requests do not have to go through the Firestore API, like the client reads and writes.

Here is the material I have referenced when creating my function:

Chained Batched writes

Get Data From Single Document Firestore Typescript

Firestore JavaScipt SDK

Returning multiple values in Tpyescript

Firestore Batched writes documentation

Here is my code that has errors:

export const updateFollowersSchema = functions
  .runWith({ timeoutSeconds: 600 })
  .https.onRequest(async (req,res) => {
    const query = await db.collection('Followers')
    query.get()
      .then(query => {
        if (query.empty){
          return null;
        } else {
          const batchArray: any[] = []
          batchArray.push(db.batch())
          let operationCounter = 0;
          let batchIndex = 0;

          query.forEach(async document => {
            const fromUserFullname = await getFromInfo(document)
            const [forUserFullname, forUserProfileImageURL] = await getForInfo(document)
            batchArray[batchIndex].update(document.ref, { 
                fromUserFullname: fromUserFullname,
                forUserFullname: forUserFullname,
                forUserProfileImageURL: forUserProfileImageURL
            })
            operationCounter++

            if(operationCounter == 499){
              batchArray.push(db.batch())
              batchIndex++
              operationCounter = 0
            }
          });

          batchArray.forEach(async batch => await batch.comit)
          return
        }
      })
  })
async function getFromInfo(data: Document){
  const userQuery = await db.collection('users').doc(data.uid).get()
  return userQuery.fullname;
}
async function getForInfo(data: Document){
  const userQuery = await db.collection('users').doc(data.uid).get()
  return [userQuery.fullname, userQuery.profileImageURL];
}

Context:

profileImageURL is a field of a user document that stores the user's profile

fullname is a field of a user document that stores the user's full name

Questions:

• Is my approach correct/feasible at all? Or is there a better approach?
• What is the syntax to pass the document data into the async functions so that the data will be grabbed from the corresponding users
• Is my approach with await and async functions going to properly handle the thread issue of having to query for the user data to populate fromUserFullname and the other objects I am writing?

Answer 1

Looks fine to me at first glance, although I would recommend changing:

batchArray.forEach(async batch => await batch.comit)

To:

Promise.all(batchArray.map(batch => batch.commit()))

So this:

• removes the typo from comit, and adds parentheses to call it.
• stops awaiting each individual transaction, allowing them to run in parallel.

Also see:

• What is the fastest way to write a lot of documents to Firestore?