How to avoid N + 1 queries in Firestore?

Question

Is there something similar to the "populate" method in Mongodb and includes in rails where Document Id's that are associated with other documents can be populated instead of me having to iterate through each document and make a separate query.

For instance let's say that I have a group document that has a collection called members. And each document in that collection corresponds to the ID of a User document. Is there any way to populate each one of those documents, without me actually making a separate query to pull the user document for each of those IDs.

Answer 1

If you want to update the contents of documents in a collection, you will have to query for each one, iterate them, and update the contents individually. There is no "update where" function in Firestore like SQL that lets you update multiple documents in a single operation.

Answer 2

There are a few options to avoiding N+1 queries in Firestore - I'm looking into testing these out further myself.

1. Collection Group Queries - you can retrieve specific sub-collections based on a name, but only one at a time. See Firestore query subcollections

2. in or array-contains-any queries - You are limited to 10 referenced documents at a time. https://firebase.googleblog.com/2019/11/cloud-firestore-now-supports-in-queries.html

3. Denormalized, Localized storage - you can store up to 1Mb in a single Firestore document, including a single text field - for example, serialized JSON. Store related document data in the same document, and update batches based on cloud functions. This would eliminate some N+1 queries by bunching up the data into the first request.

4. Secondary database - a secondary data store (ElasticSearch, Solr, or a cloud search index such as Algolia) could be used for additional functionality. You can look up referenced objects using an in type query - e.g. 'return all documents with these IDs'. The database should store identical fields in a JSON-style format. You could also use a cache (Redis or Memcache) as a temporary data store for any fast writes, e.g. aggregate calculations (SUM, MAX, MIN).

   The secondary database would need to be updated at the same time as Firestore, or via a cloud function trigger (https://firebase.google.com/docs/functions/firestore-events) that captures the changes to documents in near real-time.

   For security reasons, you probably want to do this from server-side cloud functions rather than from client side queries directly, depending on the sensitivity of the data for your application and whether it is multi-tenanted.

5. Pre-calculated queries - you could pre-calculate your data and store it in a separate document for faster access, similar in nature to a cache.

   If you wait for the first access, it would be slower to access the first time, but then you know exactly what data to cache for the 2nd, 3rd and 4th access.

   Depending on your application, you might have to store a lot of data for all potential future queries, and keep it updated when it changes.

6. Firebase - this has a different performance profile and could operate in parallel, like option #4.

I'm leaning towards options #1 / #2 (where feasible), #4 and #3 in that order, although #4 could be substantially more costly too. Hopefully options #1 and #2 (native functionality) are expanded in the future to allow more useful variations or complex scenarios.