2

I'm trying to figure out if there's a reasonable way of doing this:

My problem:

Exceeding my daily quota for reads in firestore pretty fast.

My database and what I do:

My database looks like this (simplified):

sessions: {                 // collection
    sessionId: {            // document
        users: {            // collection
            userId: {       // document
                id: string
                items: {    // collection
                    itemId: trackObject
                }
            }
        }
    }
}

Now I want to retrieve from one session, all users and their items. Most sessions have 2-3 users but some users have around 3000 items. I basically want to retrieve an array like this:

[
    {
        userId,
        items: [
            ...items
        ],
    },
    ...users
]

How I go about it currently:

So I get all users:

const usersRef = db.collection(`sessions/${sessionId}/users`);
const userSnapshots = await usersRef.get();
const userDocs = userSnapshots.docs;

Then for each user I retrieve their items:
(I use a for-loop which can be discussed but anyhow)

const user = userDocs[i].data();
const itemsRef = usersRef.collection(`${user.id}/items`);
const itemSnapshots = await itemRef.get();
const items = itemSnapshots.docs

Finally I retrieve the actual items through a map:

user.items = items.map(doc => doc.data());
return user;

My theory:

So it looks like if I do this on a session where a user has 3000 items, the code will perform 3000 read operations on firestore. After just 17 runs I eat up my 50000 operations a day.

This reasoning is somewhat based on this answer.

My question:

Is there any other way of doing this? Like getting all tracks in one read-call? Should I see if I can fit all the items into an array-key in the user-object instead of storing as a collection? Is the free version of firestore simply not designed for this many documents being retrieved in one go?

MrJalapeno
  • 1,532
  • 3
  • 18
  • 37

1 Answers1

5

If you're trying to reduce the number of document reads, you'll need to reduce the number of documents that you need to read to implement your use-case.

For example, it is fairly unlike that a user of your app will want to read the details of all 3000 items. So you might want to limit how many items you initially read, and load the additional items only on demand.

Also consider if each item needs to be its own document, or whether you could combine all items of a user into a single document. For example, if you never query the individual items, there is no need to store them as separate documents.

Another thing to consider if whether you can combine common items into a single document. An example of this is, even if you keep the items in a separate subcollection, to keep the names and ids of the most recent 30 items for a user in the user's document. This allows you to easily show a user and their 30 most recent items. Doing this you're essentially pre-rendering those 30 items of each user, significantly reducing the number of documents you need to read.

To learn more on data modeling considerations, see:

Frank van Puffelen
  • 565,676
  • 79
  • 828
  • 807