11

Expected Behavior: I assumed that when some field of a document is modified, query will only return that document and so it would be considered only 1 read. Say I query a collection with n documents, I would hope that it would be n reads for the initial state, and if a document is modified later only that document would be read again from database, resulting in n+1 reads in total.

What I see is n reads initially, and n reads later whenever one document is modified regardless of Offline Persistence being enalbed or not; resulting in 2n reads with all "fromCache : false" metadata. 

  myCollection: 
         doc1 : 
              value: "some string"
         ...
         docn: 
             value: "some text"

later I change value for one document via console:

     myCollection: 
         doc1 : 
              value: "changed to other value"
         ...
         docn: 
             value: "some text"

But I get all n documents again in the snapshot of query. Regardless of the fact that I can observe changes using "s.docChanges", it seems that snapshot has all the n documents. 

  const someCollectionRef = firebase.firestore().collection("collection/document/subdocl‌​lection"); 
  someCollectionRef.onSnapshot(s => {

      var d = [];
      s.forEach(doc => {

           d.push(doc.data()));

           //fromCache is always false even when one document is 
           //modified it seems that all documents are read from db again 
           console.log(`metadata is ` , doc.metadata);  
      }
      //d seems to have all my documents even when only 1 is modified

  });

No difference with offline persistence enabled. I tested the same example with offline persistence enabled and no difference! Modifying a document still causes all the documents to be read with "fromCache : false" metadata, i.e, they are read from database. The only time data is being read from cache is when a query is refreshed but documents are all identical.

Marcelo Glasberg
  • 29,013
  • 23
  • 109
  • 133
TheBen
  • 3,410
  • 3
  • 26
  • 51
  • If you're getting more results than expected, I'd look into that first before thinking about billing. – Doug Stevenson Jan 28 '18 at 19:02
  • 1
    My example is the plain example from the documentation. I am not getting more results than expected, but when I modify one document , the query seem to be returning all the documents in the snapshot again which then I can use snapshot.docChanges to break it into "added,modified, removed" segments. So it seems even changing one document returns the whole set of results again. I thought it should return only the changed document! Hence my question about being charged for all documents again. – TheBen Jan 28 '18 at 19:06
  • Please edit your question to illustrate exactly how you are changing the database, the result you expect, and the actual result. https://stackoverflow.com/help/how-to-ask – Doug Stevenson Jan 28 '18 at 19:09
  • I added more info. Cheers – TheBen Jan 28 '18 at 19:13
  • How do you define `someCollectionRef`? – Doug Stevenson Jan 28 '18 at 20:05
  • I have a collection/document/subcollection structure and the reference is on the subcollection like this: firebase.firestore().collection("collection/document/subdocllection") – TheBen Jan 28 '18 at 20:09
  • My experience using the Anrdoid SDK and a test that attempts to duplicate your scenario is that when the listener is attached, the first query snapshot contains 3 documents and 3 docChanges (all Add). When a document in the DB changes, the query snapshot contains 3 documents, and 1 docChange (Modified). My guess is that the unmodified documents come from the client-side cache and you are not charged for them. [This doc](https://firebase.google.com/docs/firestore/query-data/listen#view_changes_between_snapshots) describes a different scenario, but hints at use of local cache. – Bob Snyder Jan 28 '18 at 20:41
  • @BobSnyder, I logged the metadata and it doesn't seem anything is being read from Cache. Edited my question to reflect this. Thanks – TheBen Jan 28 '18 at 21:07
  • 1
    @MarcG: maybe try debugging the bytes over network to see if all the documents are being transferred or only the ones that changed. Please do post results, this is a good question – Kira Feb 15 '20 at 18:15

1 Answers1

3

I noticed this behavior and did some research: actually all is well!

When you are listening to a collection of N documents, you perform N reads for the initial state, then 1 read when a documents changes, as expected. This is described nicely in this youtube video, and in the documentation for the billing aspect.

The trap is the fromCache metadata. When you receive an update for a document change, all the other unchanged documents have fromCache: false, but they actually come from the cache, as described by a Firebase dev in this SO answer!

The fromCache metadata is true only in the scope of offline persistence, when the initial N reads were done from the local cache. When a document has received data from the server, it sets fromCache: false, and never goes back to fromCache: true. The naming is disturbing, but it seems to be working as expected.

And the code for testing:

<html>
    <body>
        <script src="/__/firebase/7.22.1/firebase-app.js"></script>
        <script src="/__/firebase/7.23.0/firebase-firestore.js"></script>
        <script src="/__/firebase/init.js"></script>
        <script>
            firebase.firestore().enablePersistence().then(() => {
                    const collectionRef = firebase.firestore().collection("/fruits");
                    collectionRef.onSnapshot({includeMetadataChanges: true}, snapshot => {
                    console.log("=================== snapshot received ===================");
                    snapshot.forEach(doc => {
                        console.log(`doc ${doc.id}, metadata:`, doc.metadata);
                    });
                });
            });
        </script>
    </body>
</html>

I modify a document using Firebase console, and in my browser's console, when the long polling request terminates, it shows that a single document was transferred:

15
[[14,["noop"]]]370
[[15,[{
  "documentChange": {
    "document": {
      "name": "projects/xxx/databases/(default)/documents/fruits/apricot",
      "fields": {
        "color": {
          "stringValue": "red"
        }
      },
      "createTime": "2020-10-13T08:14:19.649748Z",
      "updateTime": "2020-10-13T14:16:09.991161Z"
    },
    "targetIds": [
      2
    ]
  }
}
]]]122
[[16,[{
  "targetChange": {
    "resumeToken": "CgkI+bft8+Cx7AI=",
    "readTime": "2020-10-13T14:16:09.991161Z"
  }
}
]]]15
[[17,["noop"]]]
Louis Coulet
  • 3,663
  • 1
  • 21
  • 39