2

I'm using Direbase Realtime Database to store a large set of public transport informations for my mobile apps.

Last week I've tried to move everything to Firestore, that's a lot better for my database structure, thanks to his query system.

Unlike Realtime Database I cannot import data from JSON files, this has force me to use batch write operations to save the initial large block of data.

In this way I've exceeded the quota limit in a few seconds, is there a way to avoid the quota limit to import the initial data for the project?

For example in Realtime Database I was able to import a large set of data directly from the Firebase panel uploading a json file from my pc.

Marco
  • 1,057
  • 1
  • 19
  • 36

1 Answers1

3

There is currently no API nor other service to specifically bulk import data into Cloud Firestore. That means that you must use the regular API to import the data. And that unfortunately means there is currently no way to bypass quota limits to import data sets.

Frank van Puffelen
  • 565,676
  • 79
  • 828
  • 807
  • 1
    I import millions of records and that becoms really pricy and we might change the solution in the future we would really appreciate to have some server API to import millions of records in no Time and do it as bulk operation that would be really helpful – achabahe Feb 08 '18 at 18:26
  • 1
    That sounds like a great [feature request](https://firebase.google.com/support/contact/bugs-features/). – Frank van Puffelen Feb 09 '18 at 05:03
  • @FrankvanPuffelen I would argue this to be false? You can batch set new documents right? See https://stackoverflow.com/questions/46725357/firestore-batch-add-is-not-a-function – mesqueeb Apr 20 '18 at 10:27
  • Batch sets at least via the iOS SDK do not work reliably as of today. When I batch set around 300 small documents, the SDK uses up more than 1 GByte of RAM already. If I use the official limit of 500, the app crashes due to memory pressure. This as been acknowledged by Firebase support. Maybe it's better through a cloud function but I have not tried that yet. – The Lone Coder Jul 23 '18 at 13:05