0

Ok, so I am trying to create a bot that displays information through an api, that uses https://[PROJECT_ID].firebaseio/.json?shallow=trye&download=myfilename.txt , but it doesnt show all the information in the firebase, for instance this request will bring up one set of info, but then I change .json to something like "music.json" and then it gives entirely new data seperate to the other-, does anyone know how I could get all json file names to make this process alot easier?

Frank van Puffelen
  • 565,676
  • 79
  • 828
  • 807
JAricho
  • 1

1 Answers1

0

Downloading the JSON from a given path in the database should give all data under that path. I just tested this on a database of my own, and got the entire database.

If you're only seeing some of the data, it might be caused by how you process the resulting JSON file.

Frank van Puffelen
  • 565,676
  • 79
  • 828
  • 807
  • Hey, Thanks for the quick reply Frank, from what ive worked out so far .json only gives me one part of the data, I expected it to give everything but then through querying url I found that both music.json and movie.json contain new data that .json didnt have, do you know anyway I can query the firebase to get all of the different json filenames because playing the guessing game for these filenames has only worked once, meanwhile firebase docs give no help other than https://[PROJECT_ID].firebaseio/.json?download=myfilename.txt, ps sorry for the way I worded this, I dont have the correct terms – JAricho Aug 19 '21 at 18:30
  • As said, I just tested the same on a database of mine and got **all** data. How big is your exported JSON? Is it maybe going over the limits of how big a file Firebase can export in one go? I don't know what that limit is, but the database I test with exported about 3.5MB. – Frank van Puffelen Aug 19 '21 at 21:22
  • yes, it goes way over the limit, using a url I have to place shallow=true on the end just to get it to load, and trying to download after requesting only gives a small part of what I think the database could be, are you saying the reason why I cant see the music.json within .json is because the file is too large? – JAricho Aug 19 '21 at 22:36
  • If you [pass `shallow=true`](https://firebase.google.com/docs/database/rest/retrieve-data#shallow) you only get the top-level keys on the path where you request the JSON. Are you also passing `shallow=true` on the download though? I didn't see that in your question (I immediately checked for that). – Frank van Puffelen Aug 19 '21 at 22:37
  • yeah the download also includes shallow=true, so because the json database is so large + im using shallow=true to see the results its only showing parts of the database, whereas if the database was smaller I would be able to see them all? / do you know a way I can just see the main keys or query to ignore keys with the same name? – JAricho Aug 19 '21 at 23:35
  • That would've been **really** useful to put in your original question. If you specify `shallow=true`, you'll only see/get the keys directly under the path where you perform the read. Are you saying you want *some* values, not not *all* of them? That is not an option with any of the SDKs or APIs. – Frank van Puffelen Aug 19 '21 at 23:46
  • yeah I just wanted to see all of the database, those specific keys, like I said before .json gives me alot of loose data whereas music.json gave me specifics / info not within the loose data, is there not a more effective shallow thatd give me the main keys / those json filenames – JAricho Aug 20 '21 at 00:12
  • I really don't understand what you want, but I can tell you with certainty what Firebase can deliver: 1) you can get the first level of keys by getting the root (`.../.json`) and specifying `shallow=true`. 2) you can get one specific branch under that by specifying that key name (`.../music.json`) and removing `shallow=true`. 3) you can get all data by reading the root (`.../.json`) and removing `shallow=true`. But **any** request needs to stay within these limits: https://firebase.google.com/docs/database/usage/limits?authuser=0#reads – Frank van Puffelen Aug 20 '21 at 02:05
  • thank you Frank, since .json without the shallow=true cant work (the request gives back "Data requested exceeds the maximum size that can be accessed with a single request") do you know any ways to download the entire database?, maybe in parts to avoid this issue – JAricho Aug 20 '21 at 02:17
  • Yes, the link on limits I gave in my previous comment shows you what options you have: use a backup, or paginating with queries. We covered this quite a few times before, so I recommend also checking out those answers and tools linked from there: https://www.google.com/search?q=tools+to+download+large+firebase (the API hasn't substantially changed, so they should still work). – Frank van Puffelen Aug 20 '21 at 04:04