0

I have over 3,000 contacts in my phone. My firestore will hold over 1,000,000 contacts.

I can pull contacts from firestore to client to compare, I can push contact from client to firestore to compare. But I do not consider any of the two means effective.

  1. pull over 1,000,000 records from firestore to check against 3,000 records is not efficient as online data may grow to over a billion.
  2. pushing 3,000 records at 10 per request will lead to too much requests to the firestore and incur unnecessary cost.

What is the best way to compare these two data and return common elements?

Ujah Jideofor
  • 73
  • 1
  • 7
  • Can you check this [StackOverflow thread](https://stackoverflow.com/questions/54640050/firestore-comparing-two-collection-documents-is-very-slow) once? – Roopa M Jul 22 '22 at 07:18
  • Thank you Roopa M. In that thread, they were interest in finding a solution that works. I am interested in an optimal solution that reduces read and write cost. In the solution provided there, the has has to read both collections ( millions) and compare using cloud functions. – Ujah Jideofor Jul 23 '22 at 12:12

2 Answers2

1

If I were you, I will do like this way.

  1. Dumping two databases and comparing them with another database.
  2. Add one more flag for 1,000,000 contacts to know which one has the same value inside the database which has 3000 data.
  3. Upload them(1,000,000 contacts) to Firebase,
  4. To set up the filter to get querySnapshot.(Refer to Sample Code1)
  5. When you have a new contact list(from 3000 data that one)
  6. To use that new contact to filter that database(1,000,000 contacts) and remarked them with the flag 'contactExistingFlag'

Sample Code1

QuerySnapshot querySnapshot = await _firestore.collection('contactTable').where('contactExistingFlag', isEqualTo: 0).get();

//isEqualTo: 0 means that contact is new
//isEqualTo: 1 means that contact is existing

Sample Code2

QuerySnapshot querySnapshot = await _firestore.collection('contactTable').where('contactName', arrayContainsAny: ["New Member1", "New Member2"]).get();

//Use the array-contains-any operator to combine up to 10 array-contains clauses on the same field with a logical OR.
Frederic Chang
  • 509
  • 1
  • 5
  • 13
  • In the end, I may have to make over a million write for each user When I want to compare contacts. I am trying to reduce read and write rates for the function. – Ujah Jideofor Jul 21 '22 at 12:37
0

Firestore (both Cloud & Firebase) does not have any built-in operator to compare two sets. The only option is to iterate over each number and find if it has a match as recommended in this stackoverflow post. Searching for the Phone Contacts by sending them via Firebase Queries seems like a better approach.

There are ways to design your application in such a way that this kind of operation ( comparing 2 sets of numbers between address book & Firestore) is performed once in a lifetime per user signing up for an application. In future, if there is a new user getting added in Firestore database, we can do a reverse update i.e. check for that newly added number on every user's app (address book) and update that contact accordingly for given user's app if the match is found.

Note: querying the Firestore database for a matching document (with a matching phone number) does not imply billing for read operation against all documents. Please refer to the billing section of Firestore queries for more details.

Roopa M
  • 2,171
  • 4
  • 12