3

Question

In Google Apps Script, are there alternative ways to get data from external URLs?
The preferred way using UrlFetchApp.fetch(URL) doesn't work in my case.
The retrieved data (a JSON string) is cut off, probably because it's too long.

Code to show the limitation

function ShowFetchLimit() {

  var request = UrlFetchApp.fetch("http://www.gw2spidy.com/api/v0.9/json/all-items/all")
  var response = request.getContentText()              
  Logger.log('Your retrieved string length is ' response.length)

}

Preview on a public Google-Apps script

Log output

Your retrieved string length is: 10.485.277

But the real string length should be 11.439.522 characters long. It seems I'm hitting a Google Script limitation, maybe the URL Fetch POST size limit which is 10MB per call

enter image description here

DisappointedByUnaccountableMod
  • 6,656
  • 4
  • 18
  • 22
nixda
  • 2,654
  • 12
  • 49
  • 82

2 Answers2

1

you are hitting a limit that seems is unpublished but consistent with the urlFetch "post" quota.

there is really no way to workarround that Ive seen. For example lets say you instead call your server asking to write the response on a spreadsheet. then from your app you try and read the spreadsheet. ive tried reading huge sheet ranges from gas and at some point it just fails to read it. Gas has limited ram too and for one it would need to fit in ram (besides any other network read quotas).

Zig Mandel
  • 19,571
  • 5
  • 26
  • 36
1

This is actually possible provided your server supports partial downloading.

See this post for complete explanation with code :- Need alternative to UrlFetchApp.fetch for fetching large data in Apps Script

abitgcp01
  • 63
  • 6