0

I am trying to stream from firebase REST API:

I have around 500MB of data in firebase to a path, when i initiate a stream connection to it, it first wait for 2min and then end up closing connection with response pasted below.

Request: curl -Lv -X GET -H "Accept: text/event-stream" https://{URL} -o backup.txt

After trying 5 times, i have received all these response:

Response: Data requested exceeds the maximum size that can be accessed with a single request.

Response: { "error" : "Internal server error." }

My question is not about how to download all data, it is much about how to add stream Listener to a child in firebase database which has huge data in it.

When i add a listener to a path as mentioned in request, it close the connection because of all previous data present, How to achieve this?

Aman Jain
  • 895
  • 2
  • 10
  • 19
  • Possible duplicate of [Firebase error: Data requested exceeds the maximum size that can be accessed with a single request](https://stackoverflow.com/questions/38825710/firebase-error-data-requested-exceeds-the-maximum-size-that-can-be-accessed-wit) – 0mpurdy Jul 28 '17 at 08:08
  • If you read the answer carefully it says that only applies if the size <256M (which yours isn't) the second part says if it doesn't work you'll need to download it in chunks and it mentions a useful tool for doing that – 0mpurdy Jul 28 '17 at 09:34
  • @0mpurdy mentioned tool is deprecated and please check 1st link in see also: In that link they have explained about how to download gbs of data. I have edited my question, please read it again its not about downloading data, its more about streaming a path which has that huge data. – Aman Jain Jul 28 '17 at 10:34

1 Answers1

0

The error Data requested exceeds the maximum size that can be accessed with a single request would suggest that you cannot perform the request you are trying because it is too large.

You will need to split it up into multiple requests and then join them afterwards.

0mpurdy
  • 3,198
  • 1
  • 19
  • 28
  • Split it up means, do i have to request data for every child separately, can it be done in any other better way? – Aman Jain Jul 28 '17 at 07:37
  • Without knowing the shape of the data that is a very difficult question to answer – 0mpurdy Jul 28 '17 at 07:40
  • api_versions Api Response * App_Ver:17 Api Response * App_Ver:18 Api Response * App_Ver:19 Api Response * App_Ver:20 Api Response * App_Ver:21 Api Response * App_Ver:22 Api Response * App_Ver:24 Api Response * App_Ver:25 Api Response * App_Ver:26 Api Response * App_Ver:27 Api Response * App_Ver:28 api_versions is root node, with all these child, and each child have multiples child, each of them has around 1L child inside, with total of 700 MB data. – Aman Jain Jul 28 '17 at 08:03
  • This is really a separate question `How do I download all of a large Firebase database` and you could there include any code attempts you have tried so far :) This question is only really about what the error is. After checking - this is a dupe of https://stackoverflow.com/questions/38825710/firebase-error-data-requested-exceeds-the-maximum-size-that-can-be-accessed-wit – 0mpurdy Jul 28 '17 at 08:06