Managing createdAt timestamp in Firestore

Question

Every day I am importing products from external retailers into a Google Cloud Firestore database.

During that process, products can be either new (a new document will be added to the database) or existing (an existing document will be updated in the database).

Should be noted that I am importing about 10 million products each day so I am not querying the database for each product to check if it exists already.

I am currently using set with merge, which is exactly what I need as it creates a document if it doesn't exist or updates specific fields of an existing document.

Now the question is, how could I achieve a createdAt timestamp given that provided fields will be updated, therefore the original createdAt timestamp will be lost on update? Is there any way to not update a specific field if that field already exists in the document?

What about creating a new dedicated field with a Cloud Function when a doc is created. This field shall not be included in the object you pass to the `set()` method when you update the docs. — Renaud Tarnec, Aug 02 '18 at 15:01
I've elaborated an answer based on my comment, with the corresponding Cloud Function code. Hope this helps. — Renaud Tarnec, Aug 02 '18 at 15:34
On the Node.js side, the [DocumentSnapshot](https://cloud.google.com/nodejs/docs/reference/firestore/0.15.x/DocumentSnapshot#createTime) contains a field for `createTime`. — Bob Snyder, Aug 02 '18 at 15:58
@BobSnyder Thanks for the info Bob, interesting to know! Do you suggest that the CF could use `snap.createTime;` as the field value? — Renaud Tarnec, Aug 02 '18 at 16:00
@RenaudTarnec: It's not clear to me. The documentation for the Admin version of `DocumentSnapshot` is different than `@google-cloud/firestore`. Not sure if `createTime` is available. — Bob Snyder, Aug 02 '18 at 16:08
@BobSnyder See my update to the answer: it works. No idea which method is the best to use however! — Renaud Tarnec, Aug 02 '18 at 16:11

Renaud Tarnec · Accepted Answer · 2018-08-02T20:05:49.660

7

I suggest that you use a Cloud Function that would create a new dedicated field when a Firestore doc is created. This field shall not be included in the object you pass to the set() method when you update the docs.

Here is a proposal for the Cloud Function code:

const functions = require('firebase-functions');

const admin = require('firebase-admin');

admin.initializeApp();


exports.productsCreatedDate = functions.firestore
  .document('products/{productId}')
  .onCreate((snap, context) => {

    return snap.ref.set(
      {
        calculatedCreatedAt: admin.firestore.FieldValue.serverTimestamp()
      },
      { merge: true }
    )
    .catch(error => {
       console.log(error);
       return false;
    });
  });

Based on Bob Snyder comment above, note that you could also do:

const docCreatedTimestamp = snap.createTime;
return snap.ref.set(
  {
    calculatedCreatedAt: docCreatedTimestamp
  },
  { merge: true }
)
.catch(...)

edited Aug 02 '18 at 20:05

answered Aug 02 '18 at 15:33

Renaud Tarnec

79,263
10
95
121

That looks indeed like what I was after, I'll do some testing tonight and accept the answer if confirmed. Thanks – SebScoFr Aug 02 '18 at 15:54
Cool. I am interested to know how it behaves when you inject 10 millions document. – Renaud Tarnec Aug 02 '18 at 15:58
It seems to do the job. It behaves quite nicely, especially when using batch writes. In the end it comes down to how fast you can send documents to it. – SebScoFr Aug 03 '18 at 07:58
Be very careful with this, updatedAt, for example, will create an infinite loop! – Jonathan Apr 01 '20 at 01:02
https://stackoverflow.com/a/60963531/271450 - In case you're looking for updatedAt – Jonathan Apr 01 '20 at 04:20

score 4 · Answer 2 · answered Oct 22 '21 at 15:14

4

in version firebase 9 the correct way is:

import { serverTimestamp } from "firebase/firestore";
....
return snap.ref.set(
      {
        calculatedCreatedAt: serverTimestamp();
.....

answered Oct 22 '21 at 15:14

luis urdaneta

91
1
4

Managing createdAt timestamp in Firestore

2 Answers2

Linked