How should I structure Firestore documents with a large number of docs needed to be read each time

Question

I have a list of service that each have a number of fields that are structured like this

Service1*

     ServiceID*
     NetID*
     Country*
     price*
     xxx*
     xxx*
     10 fields in total*

Service2*

     ServiceID*
     NetID*
     Country*
     price*
     xxx*
     xxx*
     10 fields in total*

I have arranged currently where each service is its own document for a total of 1,180. However when a user visits my site and wants to select a service, the server would fetch all 1180. With just 100users firebase would reading 100k docs. I was thinking of saving a copy of all the services as a csv and loading it from there every time a user wants to search the service.

Is there any better of structuring the data or using another method to reduce my read count of Firestore.

Alex Mamo · Accepted Answer · 2019-10-28T13:45:10.380

2

Is there any better of structuring the data

There is no other better method for doing that.

However when a user visits my site and wants to select a service, the server would fetch all 1180.

That's not a proper way to interact with Cloud Firestore. You are downloading huge ammout of data with a high cost. Beside that, a user will never need all that amount of data at once. A solution for that will be to load data in smaller chunks. For Android, I recommend you see my answer from the following post:

How to paginate Firestore with Android?

I was thinking of saving a copy of all the services as a csv and loading it from there every time a user wants to search the service.

There is no need for that. Firestore already has its own caching mechanism. So once you get a document, it will always be read from cache, as long as on the server, no new changes are made. More infos here:

https://firebase.google.com/docs/firestore/manage-data/enable-offline

For the web, offline persistence is disabled by default. To enable persistence, call the enablePersistence method. Cloud Firestore's cache isn't automatically cleared between sessions. Consequently, if your web app handles sensitive information, make sure to ask the user if they're on a trusted device before enabling persistence.

edited Oct 28 '19 at 13:45

answered Oct 28 '19 at 11:23

Alex Mamo

130,605
17
163
193

I forgot to add that this was for a website – Abdiaziz Ahmed 3L4G Oct 28 '19 at 11:49
I think maybe that I could separate the data into more collections then the user can sort through those collections to find their service – Abdiaziz Ahmed 3L4G Oct 28 '19 at 11:50
No, that;s not an option. You cannot query accros different collections, unles you use collection group query. But for your use is not needed. A single collection is enough. – Alex Mamo Oct 28 '19 at 11:51
The way I was thinking was having a user select from a list of Collection names and then retrieving its documents I could cut the number retrieved from 1000 to 100 – Abdiaziz Ahmed 3L4G Oct 28 '19 at 11:53
You cannot iterate through collections. So the initial solution will solve the problem. – Alex Mamo Oct 28 '19 at 11:55

How should I structure Firestore documents with a large number of docs needed to be read each time

1 Answers1

Linked

Related