How to come around Firebase Realtime Database's server side timestamp volatility when one wants to compare timestamps on updates?

Question

After reading the docs on ServerValue.TIMESTAMP, I was under the impression that once the object hits the database, the timestamp placeholder evaluates once and remains the same, but this was not the case for me:

// Example on Node:

> const db = f.FIREBASE_APP.database();
> const timestamp = f.FIREBASE_APP.database.ServerValue.TIMESTAMP;

> const ref = db.ref('/test'); 

> ref.on(
... 'child_added',
... function(snapshot) {
..... console.log(`Timestamp from listener: ${snapshot.val().timestamp}`);
..... }
... )

> var child_key = "";

> ref.push({timestamp: timestamp}).then(
... function(thenable_ref) {
..... child_key = thenable_ref.key;
..... }
... );
Timestamp from listener: 1534373384299

> ref.child(child_key).once('value').then(
... function(snapshot) {
..... console.log(`Timestamp after querying: ${snapshot.val().timestamp}`);
..... }
... );
> Timestamp after querying: 1534373384381

> 1534373384299 < 1534373384381
true

The timestamp is different when queried from the on listener and it is different during a later query.

Is this like this by design and I just missed some parts of the documentation? If this is the case, when does the ServerValue.TIMESTAMP stabilize?

I am building a CQRS/ES library on the Realtime Database, and just wanted to avoid the expected_version (or sequence numbers) of events.

UPDATE

The proof for Frank's explanation below:

/* `db`, `ref` and `timestamp` are defined above,
   and the test path ("/test") has been deleted
   from DB beforehand to avoid noise.
*/

> ref.on(   
... 'child_added', 
... function(snapshot) {    
..... console.log(`Timestamp from listener: ${snapshot.val().timestamp}`);     
..... }
... )

> ref.on(   
... 'value', 
... function(snapshot) {
..... console.log(snapshot.val());
..... }
... )

> ref.push({timestamp: timestamp}); null;

Timestamp from listener: 1534434409034
{ '-LK2Pjd8FS_L8hKqIpiE': { timestamp: 1534434409034 } }
{ '-LK2Pjd8FS_L8hKqIpiE': { timestamp: 1534434409114 } }

Bottom line is, if one needs to rely on immutable server side timestamps, keep this in mind, or work around it.

It's not supposed the work the way you're presenting it here. Data doesn't automatically change in the database after it's been written, until it's written again. If it did work like this, it would completely break the expectations of how listeners work. — Doug Stevenson, Aug 16 '18 at 00:30
@DougStevenson I am about to test Frank's explanation, but it makes sense what he says as the values are indeed different on subsequent queries (i.e., at the `on` listener, and after it). I am trying to use Firebase's features as idiomatically as possible, but this issue did baffle me, and I needed to work around it. (It took me only about 10 minutes and not many people would use the DB this way, but it was still an unpleasant surprise.) Please let me know if I am doing something wrong. — toraritte, Aug 16 '18 at 15:24
Ah, yes, Frank is correct. You're getting an estimate first, then the actual value later. It shouldn't change after that. — Doug Stevenson, Aug 16 '18 at 15:43

score 1 · Answer 1 · answered Aug 16 '18 at 06:14

When you perform the ref.push({timestamp: timestamp}) the Firebase client immediately makes an estimate of the timestamp on the client and fires an event for that locally. It then send the command off to the server.

Once the Firebase client receives the response from the server, it checks if the actual timestamp is different from its estimate. If it is indeed different, the client fires reconciliatory events.

You can most easily see this by attaching your value listener before setting the value. You'll see it fire with both the initial estimates value, and the final value from the server.

Also see:

Are saying that this is by design and there is nothing that can be done? Shouldn't this be documented in this case? There are probably other scenarios where comparing server side timestamps for equality is important, but this would break all of them. — toraritte, Aug 16 '18 at 15:39
Is [Firestore's `serverTimestamp`](https://firebase.google.com/docs/reference/js/firebase.firestore.FieldValue#.serverTimestamp) implemented the same way? — toraritte, Aug 16 '18 at 16:05

toraritte · Answer 2 · 2018-08-17T22:11:53.073

CAVEAT: After wasting another day, the ultimate solution is not to use Firebase server timestamps at all, if you have to compare them in a use case that is similar to the one below. When the events come in fast enough, the second 'value' update may not trigger at all.

One solution, to the double-update condition Frank describes in his answer, is to get the final server timestamp value is (1) to embed an on('event', ...) listener inside an on('child_added', ...) and (2) remove the on('event', ...) listener as soon as the specific use case permits.

> const db = f.FIREBASE_APP.database();
> const ref = db.ref('/test');
> const timestamp = f.FIREBASE_APP.database.ServerValue.TIMESTAMP;

> ref.on(
    'child_added',
    function(child_snapshot) {
      console.log(`Timestamp in 'child_added':    ${child_snapshot.val().timestamp}`);
      ref.child(child_snapshot.key).on(
        'value',
        function(child_value_snapshot) {

          // Do a timestamp comparison here and remove `on('value',...)`
          // listener here, but keep in mind: 
          // + it will fire TWICE when new child is added
          // + but only ONCE for previously added children!

          console.log(`Timestamp in embedded 'event': ${child_value_snapshot.val().timestamp}`);
          }
        )
      }
    )

// One child was already in the bank, when above code was invoked:
Timestamp in 'child_added':    1534530688785
Timestamp in embedded 'event': 1534530688785

// Adding a new event:
> ref.push({timestamp: timestamp});null;

Timestamp in 'child_added':    1534530867511
Timestamp in embedded 'event': 1534530867511
Timestamp in embedded 'event': 1534530867606

In my CQRS/ES case, events get written into the "/event_store" path, and the 'child_added' listener updates the cumulated state whenever new events come in, where each event has a ServerValue.TIMESTAMP. The listener compares the new event's and the state's timestamp whether the new event should be applied or it already has been (this mostly matters when the server has been restarted to build the internal in-memory state). Link to the full implementation, but here's a shortened outline on how single/double firing has been handled:

event_store.on(
    'child_added',
    function(event_snapshot) {

        const event_ref = event_store.child(event_id)

        event_ref.on(
            'value',
            function(event_value_snapshot){

                const event_timestamp = event_value_snapshot.val().timestamp;

                if ( event_timestamp <= state_timestamp ) {

                    // === 1 =======
                    event_ref.off();
                    // =============

                } else {

                    var next_state =  {};

                    if ( event_id === state.latest_event_id ) { 
                        next_state["timestamp"] = event_timestamp;

                        Object.assign(state, next_state);
                        db.ref("/state").child(stream_id).update(state);

                        // === 2 =======
                        event_ref.off();
                        // =============

                    } else {

                        next_state =  event_handler(event_snapshot, state);

                        next_state["latest_event_id"] = event_id;
                        Object.assign(state, next_state);
                    }
                }
            }
        );
    }
);

When the server is restarted, on('child_added', ...) goes through all events already in the "/event_store", attaching on('value',...) dynamically on all children and compares the events` timestamps to the current state's.

If the event is older than the age of the current state (event_timestamp < state_timestamp is true), the only action is detaching the 'value' listener . This callback will be fired once as the ServerValue.TIMESTAMP placeholder has already been resolved once in the past.
Otherwise the event is newer, which means that it hasn't been applied yet to the current state and ServerValue.TIMESTAMP also hasn't been evaluated yet, causing the callback to fire twice. To handle the double update, this block saves the actual child's key (i.e., event_id here) to the state (to latest_event_id) and compares it to the incoming event's key (i.e., event_id):

How to come around Firebase Realtime Database's server side timestamp volatility when one wants to compare timestamps on updates?

2 Answers2