41

Was wondering if there are any common practices in backup up a firebase DB. My concern is some process accidentally wiping out our Database.

Thanks!

BigPoppa
  • 1,205
  • 2
  • 13
  • 21

5 Answers5

45

As of the time of this question, Firebase backs up all instances daily. So while keeping your own backups may still be useful, it's not essential.

To create your own backups, you can simply curl the data:

curl https://<instance>.firebaseio.com/.json?format=export

Note that for multiple gigabytes of data, this will slow things down and lock read access for a short period. It would be better in this case to chunk the backups and work with smaller portions. The shallow parameter can help here by providing a list of keys for any given path in Firebase, without having to fetch the data first.

curl https://<instance>.firebaseio.com/.json?shallow=true

As previously mentioned, there are also several GitHub libs available for this, and incremental backups are practical with some creativity and a worker thread on the real-time SDK.

Kato
  • 40,352
  • 6
  • 119
  • 149
  • Clarification: It seems the auto backup is only there for the "Bonfire" plan and above currently https://www.firebase.com/blog/2015-03-05-private-backups-for-firebase-data.html – pjco Jun 01 '15 at 15:32
  • 6
    Incorrect. You're referring to private backups which are pushed out to a GCS or S3 bucket. Firebase backs up all instances daily for all plans, including free, as noted in my answer. – Kato Jun 01 '15 at 17:47
  • Are you saying you back them up ***internally***? Because that's not the same. Having to contact support to restore a backup is something you should clarify in your post.... unless I am missing some functionality, in that case, please point to it. Your answer as written is misleading. – pjco Jun 01 '15 at 18:01
  • 6
    Your downvote is noted. Have a great day. – Kato Jun 01 '15 at 23:24
  • 3
    Only downvoted because you aren't giving the complete answer, I would happily change it if you didnt evade the question with snarky little comments. I came to this question looking for the answer and am still wondering what the difference is between private backups and what you describe. Why cant you just say? Are you really the dev evangelist at firebase? (You are behaving awfully unprofessionally). – pjco Jun 01 '15 at 23:37
  • 9
    There is nothing misleading or dishonest about the answer here. It explains how to create one's own backups, which was the question. It notes that Firebase provides backups of data. As of the time of writing the answer, private backups did not exist. Your opinion is noted and I've considered it. I appreciate the feedback. Feel free to post your own answer if you're unhappy with this one. No hard feelings. No attitudes required. – Kato Jun 02 '15 at 16:23
  • 1
    Fair enough, if you clarified how one would go about restoring a daily backup, I personally think that could improve the answer. Cheers. – pjco Jun 02 '15 at 16:30
  • Thanks @kato for this answer. I've used this with nodejs to perform daily backup. https://github.com/hardiksondagar/firebase-backup. Again thanks :-) – Hardik Sondagar Feb 22 '16 at 06:04
  • Is this still the case as of today (after the migration to the Google Cloud)? So if someone fat fingered and deleted my entire DB tonight, I could e-mail Firebase support and have it restored to last night's state? How many days does this internal Firebase back-up go back for? – Lloyd Banks Jun 16 '16 at 03:35
  • It's still the same database and we're still Firebase. Integrating with GCP and adding more features didn't involve removing common sense. We still have disaster recovery backups, as well as [Private Backups](https://firebase.googleblog.com/2015/03/private-backups-for-firebase-data_21.html), and we still make them available to you as necessary. – Kato Jun 17 '16 at 19:31
  • How do you log in with CURL? – Pier Sep 13 '16 at 04:42
  • 3
    @Kato: Is the answer still valid as of today, in June, 2017? Because from [this](https://firebase.google.com/docs/database/web/backups) link it seems 'automated daily backups' are available only for the 'Blaze' plan, not the free plan. – Piyush Soni Jun 07 '17 at 17:54
  • @Kato while backing up data from firebase, it return { "error" : "Internal server error." } request: curl https://{URL}?format=export -o backup.db – Aman Jain Jul 28 '17 at 07:14
  • See my answer above. Firebase Firestore now support it natively. – Gal Bracha Aug 10 '18 at 10:33
  • I doubt it is possible to authenticate this way, so this solution doesn't work unless the whole database is accessible to read by all, which might not be advisable in many cases... – mrtnmgs Sep 03 '18 at 10:23
  • Auth via REST API is covered in the doc and possible. – Kato Sep 11 '18 at 02:28
15

There are now "Import Data" and "Export Data" buttons on the data page of the web interface for every project, so you can now backup your data with a button click!

Import and Export buttons

c.fogelklou
  • 1,781
  • 1
  • 13
  • 26
  • 1
    Hmm, not seeing these in the database tab. Where is that screenshot taken? – Patrick Apr 17 '20 at 15:41
  • Looks like now you need to click on the three vertical dots up in the right hand corner to see export and import. – c.fogelklou Apr 21 '20 at 05:49
  • Ah, just found the three dots in the Realtime DB. Looks like there's no option to import/export for Firestore. – Patrick Apr 21 '20 at 21:41
  • @Patrick Firebase's official documentation recommends their backup service through GCP for Firestore: https://firebase.google.com/docs/firestore/manage-data/export-import. You can automate the CLI commands with a scheduled CRON job. The back-up files are not JSON, but instead some manipulation of LevelDB output. – Sean Perkins Apr 21 '20 at 23:52
  • @SeanPerkins yeah, saw that. But ugh all that just to export json? Really, Google? I'd take a one-button solution over that any day. – Patrick Apr 22 '20 at 13:48
9

just yesterday wrote a shell-script, which utilizes firebase-tools (npm install -g firebase-tools), in order to have these database dumps contained within my regular backup cronjob:

#!/bin/bash
# $1 is the Firebase projectId.
# $2 is the destination directory. 
# example usage: cron_firebase.sh project-12345 /home/backups/firebase
# currently being triggered by /etc/cron.hourly/firebase-hourly.cron
PROJECTID=$1
DESTINATION=$2
FIREBASE="$(which firebase)"
NOW="$(date +"%Y-%m-%d_%H%M")"
cd $DESTINATION
$FIREBASE --project $PROJECTID database:get / > ./$PROJECTID.$NOW.json
tar -pczf $PROJECTID.$NOW.tar.gz ./$PROJECTID.$NOW.json && rm ./$PROJECTID.$NOW.json

update: in the meanwhile, one can auto backup to Google Cloud Storage Bucket

...goto Firebase Console -> Realtime Database -> and click tab Backups.

Martin Zeitler
  • 1
  • 19
  • 155
  • 216
6

It is now possible to backup and restore Firebase Firestore using Cloud Firestore managed export and import service

You do it by:

  1. Create a Cloud Storage bucket for your project:

  2. Set up gcloud for your project using gcloud config set project [PROJECT_ID]

EXPORT

Export all by calling gcloud alpha firestore export gs://[BUCKET_NAME] Or Export a specific collection using gcloud alpha firestore export gs://[BUCKET_NAME] --collection-ids='[COLLECTION_ID_1]','[COLLECTION_ID_2]'

IMPORT

Import all by calling gcloud alpha firestore import gs://[BUCKET_NAME]/[EXPORT_PREFIX]/ where [BUCKET_NAME] and [EXPORT_PREFIX] point to the location of your export files. For example - gcloud alpha firestore import gs://exports-bucket/2017-05-25T23:54:39_76544/

Import a specific collection by calling: gcloud alpha firestore import --collection-ids='[COLLECTION_ID_1]','[COLLECTION_ID_2]' gs://[BUCKET_NAME]/[EXPORT_PREFIX]/

Full instructions are available here: https://firebase.google.com/docs/firestore/manage-data/export-import

Community
  • 1
  • 1
Gal Bracha
  • 19,004
  • 11
  • 72
  • 86
  • 1
    Could you expand on this answer by including some of the key features mentioned on that page *in the answer itself*? As it stands, if the URL changes or goes down this answer would effectively become useless. – Michael Dodd Aug 10 '18 at 10:07
  • 1
    Thank you. @MichaelDodd I've added more info. See if that is more useful now. – Gal Bracha Aug 10 '18 at 10:32
  • Can you show how to schedule this using Firebase Cron? https://firebase.googleblog.com/2019/04/schedule-cloud-functions-firebase-cron.html – aaronvargas Jun 17 '19 at 18:40
  • 1
    @aaronvargas See this blog post to get your answer https://blog.emad.in/automate-firestore-db-backups/ :) – eMAD Sep 16 '19 at 19:20
4

Just to expand @kato's answer using curl.

I was looking for ways to run the command every night. My solution:
1) created a compute engine (basically a VM) in Google Cloud. You might be familiar with EC2 if you are from AWS world.
2) Wrote a simple cronjob, something like this

0 23 * * * /usr/bin/curl https://yourdatabaseurl.com/.json?format=export -o /tmp/backuptest_`date +\%d\%m\%y`.bk

I am sure there might be a simpler way to do this within the free tier itself. Like using cloud functions.

viggy28
  • 760
  • 1
  • 10
  • 21