How to delete a large node in firebase

Question

I have a Firebase child node with about 15,000,000 child objects with a total size of about 8 GB of data.

exampele data structure:

firebase.com/childNode/$pushKey

each $pushKey contains a small flat dictionary: 
{a: 1.0, b: 2.0, c: 3.0}

I would like to delete this data as efficiently and easy as possible. How?

What i Tried: My first try was a put request:

PUT firebase.com/childNode.json?auth=FIRE_SECRET
data-raw: null

response: { "error": "Data requested exceeds the maximum size that can be accessed with a single request. Contact support@firebase.com for help." }

So that didn't work, let's do a limit request:

PUT firebase.com/childNode.json?auth=FIRE_SECRET&orderBy="$key"&limitToFirst=100
data-raw: null

response: { "error": "Querying related parameters not supported on this request type" }

No luck so far :( What about writing a script that will get the first X number of keys and then create a patch request with each value set to null?

GET firebase.com/childNode.json?auth=FIRE_SECRET&shallow=true&orderBy="$key"&limitToLast=100

{ "error" : "Mixing 'shallow' and querying parameters is not supported" }

It's really not going to be easy this one? I could remove the shallow requirement and get the keys, and finish the script. I was just hoping there would be a easier/more efficient way???

Another thing i tried were to create a node script that listen for childAdded and then directly tries to remove those children?

ref.authWithCustomToken(AUTH_TOKEN, function(error, authData) {
  if (error) {console.log("Login Failed!", error)}
  if (!error) {console.log("Login Succeeded!", authData)}

  ref.child("childNode").on("child_added", function(snap) {
    console.log(`found: ${snap.key()}`)
    ref.child("childNode").child(snap.key()).remove( function(err) {
      if (!err) {console.log(`deleted: ${snap.key()}`)}
    })
  })
})

This script actually hangs right now, but earlier I did receive somethings like a max stack limit warning from firebase. I know this is not a firebase problem, but I don't see any particular easy way to solve that problem.

score 1 · Answer 1 · answered Nov 27 '15 at 01:05

Downloading a shallow tree, will download only the keys. So instead of asking the server to order and limit, you can download all keys.

Then you can order and limit it client-side, and send delete requests to Firebase in batches.

You can use this script for inspiration: https://gist.github.com/wilhuff/b78e7391396e09f6c614

score 0 · Answer 2 · answered Jan 19 '17 at 09:20

0

Use firebase cli tool for this: firebase database:remove --project .

answered Jan 19 '17 at 09:20

AshuKingSharma

757
7
20

If I remember correctly then the challenge was only to delete a node and not a complete project. – Andreas Okholm Jan 26 '17 at 09:51
@AndreasOkholm: sorry, my bad it is remove --project – AshuKingSharma Jan 26 '17 at 18:25
1

FYI, older CLI versions couldn't delete large nodes, but the latest versions can. – Joseph Lust Sep 06 '19 at 18:06

score 0 · Answer 3 · answered Dec 23 '20 at 19:41

0

In Browser Console this is fastest way

database.ref('data').limitToFirst(10000).once('value', snap => { var updates = {}; snap.forEach(snap => { updates[snap.key] = null; }); database.ref('data').update(updates); });

answered Dec 23 '20 at 19:41

Valentin Vedic

1

How to delete a large node in firebase

3 Answers3

Linked