How to automatically backup Firestore database daily

Question

I would like to backup a Firestore database on a daily basis.

My data structure is like this:

usersCollection
  uid
     defaultCurrency: 'USD'
     name: 'something'
     dreamsCollection
                     name
                     image

I have looked at firestore-export-import and node-firestore-backup to do the backup and export the data to a JSON file.

My questions are:

If I create a cloud function how do I run the cloud function daily?
What is the best way to do the backup?
- Should I store the data in firebase storage and how to do this?
- Should I save the JSON to a separate service etc?

Any help, no matter how small would be appreciated.

Thanks

For #1, see https://stackoverflow.com/questions/42790735/cloud-functions-for-firebase-trigger-on-time — Frank van Puffelen, May 15 '18 at 13:43
Possible duplicate of [Is it possible to backup Firebase DB?](https://stackoverflow.com/questions/27910784/is-it-possible-to-backup-firebase-db) — Martin Zeitler, Nov 16 '18 at 14:03
Does this answer your question? [Cloud Functions for Firebase trigger on time?](https://stackoverflow.com/questions/42790735/cloud-functions-for-firebase-trigger-on-time) — My Car, Feb 04 '23 at 02:47

eMAD · Answer 1 · 2021-08-31T13:47:10.190

14

I honestly don't like the sample solution provided on the firebase page. It's unnecessarily convoluted especially the part where you need to run things on app engine ¯\_(ツ)_/¯

There is a much simpler way to do this using the Cloud Scheduler and google-auth-library, to directly store the backups in the Firebase Cloud Storage. I have written an in depth article with the seamlessly working code that I'm using in production. Have a look.

https://blog.emad.in/automate-firestore-db-backups/

edited Aug 31 '21 at 13:47

answered Sep 16 '19 at 19:17

eMAD

473
4
6

Awesome! Maybe a silly follow-up question, how would one restore the firestore snapshot from the backup ? – Sebastien FERRAND Mar 23 '22 at 17:32
1

@SebastienFERRAND I came across your question while trying to find an answer to the same question for myself (how to restore a Firestore backup). The Firebase documentation encourages you to load it in BigQuery, but that will not work if you are exporting all of your collections. Instead you can use the Firebase Import/Screen screen in the GCP console to re-import the data. You can either overwrite your existing Firestore, or you can create a new Firebase project and load it there for viewing. – most200 May 12 '22 at 21:36

uksz · Answer 2 · 2019-04-09T07:21:07.023

12

You can now export and import the data to the specified bucket.

More info here: https://firebase.google.com/docs/firestore/manage-data/export-import

Here is an info on how to do it automatically:

https://firebase.google.com/docs/firestore/solutions/schedule-export

edited Apr 09 '19 at 07:21

answered Nov 29 '18 at 12:15

uksz

18,239
30
94
161

3

IMHO it's not a good solution. It should be a simple Cloud Function to deploy, not needing to use App Engine. – Jeremy Belolo Aug 30 '19 at 07:21
Also, this solution doesn't work. The permissions for Firestore are never approved for AppEngine to connect.. – Oliver Dixon Apr 20 '20 at 19:45
1

I don't understand the above comments. Just following : https://firebase.google.com/docs/firestore/solutions/schedule-export does exactly what I want. I used the console path, pasted the code, created a bucket where to store to, setup the job, gave the rights and it works. Added a retention ploicy and a rule to clean up, done. – Vincent Gerris May 26 '20 at 13:02

score 8 · Answer 3 · answered Feb 16 '21 at 09:46

I've blogged about creating a fully managed, serverless automatically triggered Workflow that triggers the Firestore export/backup API, and it places into a Cloud Storage bucket, this way you ensure proper backup for disaster recovery.

As it’s serverless no maintenance of SDK tools, no updates to libraries are involved, and even a non-developer can set it up.

Full article is here: Firestore Backups the easy way with Cloud Workflows

Venryx · Answer 4 · 2021-10-06T00:10:19.457

Approach 1 (App Engine)

I've been using this solution for months, and it works well: https://github.com/firebase/snippets-node/tree/master/firestore/solution-scheduled-backups

This sample demonstrates using AppEngine cron jobs to run nightly backups of data in Cloud Firestore.

Approach 2 (Cloud Scheduler)

That said, the approach described by @eMAD looks good as well; it's based on the same gcloud command, but the Cloud Scheduler it uses may be preferred over the App Engine scheduling. (for example, it lets you specify an exact time of day, instead of it just being the time at which you first set up the schedule)

There's also this helper package for setting up Cloud-Scheduler-based automated backups.

EDIT: I now use the helper-package mentioned above (simple-firestore-backup), as the npm-package helps keep the implementation consistent between projects, and I like it over the app-engine solution because cloud-functions have more flexibility.

Note however that the readme is currently "missing" a few steps (depending on what you consider "self-evident" or not). If you want a more comprehensive set of instructions, augment the readme with the additional steps I explain here.

score 0 · Answer 5 · answered Aug 10 '23 at 09:59

0

A new feature in Firebase might be of help with backups up to 7 days: Point-in-time recovery.

https://cloud.google.com/firestore/docs/pitr

answered Aug 10 '23 at 09:59

Alexandru Pavaloi

19
2

How to automatically backup Firestore database daily

5 Answers5

Approach 1 (App Engine)

Approach 2 (Cloud Scheduler)

Linked