15

I'm making a simple app that informs a client that other clients clicked a button. I'm storing the clicks in a Firebase (db) using:

db.push({msg:data});

All clients get notified of other user's clicks with an on, such as

db.on('child_added',function(snapshot) { 
  var msg = snapshot.val().msg; 
});

However, when the page first loads I want to discard any existing data on the stack. My strategy is to call db.once() before I define the db.on('child_added',...) in order to get the initial number of children, and then use that to discard that number of calls to db.on('child_added',...).

Unfortunately, though, all of the calls to db.on('child_added',...) are happening before I'm able to get the initial count, so it fails.

How can I effectively and simply discard the initial data?

mix
  • 6,943
  • 15
  • 61
  • 90
  • See also: [How to retrieve only new data](http://stackoverflow.com/questions/18270995/how-to-retreive-only-new-data) – Kato Jan 16 '15 at 16:06
  • 1
    Since the common use case for this question is creating a message queue, the best answer is to use a [message queue strategy](https://github.com/firebase/firebase-work-queue), where events are added and deleted from a queue, and not bother with this added complexity. Please consider that route before any of the answers below. – Kato Jan 31 '15 at 17:40

2 Answers2

26

For larger data sets, Firebase now offers (as of 2.0) some query methods that can make this simpler.

If we add a timestamp field on each record, we can construct a query that only looks at new values. Consider this contrived data:

{
  "messages": {
     "$messageid": {
        "sender": "kato",
        "message": "hello world"
        "created": 123456  // Firebase.ServerValue.TIMESTAMP
     }
  }
}

We could find messages only after "now" using something like this:

var ref = new Firebase('https://<your instance>.firebaseio.com/messages');

var queryRef = ref.orderBy('created').startAt(Firebase.ServerValue.TIMESTAMP);

queryRef.on('child_added', function(snap) {
  console.log(snap.val());
});
Kato
  • 40,352
  • 6
  • 119
  • 149
  • 1
    How would you deal with inconsistency if I create a new record myself with REST API and set `created` timestamp as my server's timestamp? Which is ofcourse UTC too, but you know it may not match exactly Firebase's time – Skyzer May 29 '15 at 00:37
  • 3
    Use Firebase.ServerValue.TIMESTAMP and there won't be any inconsistencies. If you're worried about someone hacking your software to set inconsistent times--they must really be bored--then use security rules to enforce the timestamps consistency. – Kato May 29 '15 at 17:54
  • 1
    Thanks alot, I looked now again, REST API does indeed support setting Firebase timestamp `'{".sv": "timestamp"}'` https://www.firebase.com/docs/rest/api/#section-server-values – Skyzer May 29 '15 at 22:42
  • 1
    hello kato, your above example doesnt work, `orderBy` does not exist in firebase's javascript lib, only in REST API. Also it seems that firebase timestamp (which is just json object) cannot be passed to `startAt`, in the example docs, `startAt` was taking only integers. Please refer to my updated question http://stackoverflow.com/questions/30493719/get-only-new-child-added-after-page-load – Skyzer May 31 '15 at 04:52
  • 1
    This question was asked over a year ago. The API has changed since then. See [queries](https://www.firebase.com/docs/web/guide/retrieving-data.html#section-queries) and any of the orderBy*() methods. – Kato Jun 01 '15 at 05:26
  • Okay, thank you, but still I have tried all possibilities, and none of them are returning new messages. `notifRef.startAt(Firebase.ServerValue.TIMESTAMP).on('child_added', function(snap) {;` this was only one that didn't throw exception, but when I'm adding new messages with REST and using Firebase's timestamp, no new `child_added` event is fired in browser. In firebase console new message is added though. Please look at my **edit2** section in my question http://stackoverflow.com/questions/30493719/get-only-new-child-added-after-page-load – Skyzer Jun 01 '15 at 10:06
  • Completely unrelated to this thread and a duplicate question answered several times on SO. Please stop soliciting. – Kato Jun 01 '15 at 17:50
  • Kato, got it working! Thanks alot for inputs. The confusing part is if I either save new notifications with by key, or just setting. So it's either `push` or `set`. If either of them is used, queries have to be done completely different. – Skyzer Jun 08 '15 at 00:13
17

If I understand your question correctly, it sounds like you only want data that has been added since the user visited the page. In Firebase, the behavior you describe is by design, as the data is always changing and there isn't a notion of "old" data vs "new" data.

However, if you only want to display data added after the page has loaded, try ignoring all events prior until the complete set of children has loaded at least once. For example:

var ignoreItems = true;
var ref = new Firebase('https://<your-Firebase>.firebaseio.com');
ref.on('child_added', function(snapshot) {
  if (!ignoreItems) {
    var msg = snapshot.val().msg;
    // do something here
  }
});
ref.once('value', function(snapshot) {
  ignoreItems = false;
});

The alternative to this approach would be to write your new items with a priority as well, where the priority is Firebase.ServerValue.TIMESTAMP (the current server time), and then use a .startAt(...) query using the current timestamp. However, this is more complex than the approach described above.

Rob DiMarco
  • 13,226
  • 1
  • 43
  • 55
  • yes, that's what I'm looking for. I was trying a variation on that but mine was more complicated. thx! – mix Nov 09 '13 at 23:26
  • This solution requires the data be downloaded twice which is non-ideal for large data sets. Also, if the data set is large enough, value will take a long time to load and some items in child_added could be missed. – Keith Carter Jan 17 '15 at 02:23
  • It is true that some items could be missed if the data set is sufficiently large, but it does not cause the data to be downloaded twice. In general, if only querying for "new items since a particular client came online", it is a better bet to query data based upon timestamp, or use a pub-sub model. – Rob DiMarco Jan 17 '15 at 06:41
  • Yes, I do the same thing. I do a chat application, and have a ChatManager singleton to be the first handler of firebase events then notify other listeners by using Observer design pattern. When the ChatManager init, I wait for 4-5s before turn on `should_handle_event` variable. Any event comes before that will be discard. – Tran Quan Feb 09 '18 at 01:09