1

Given a datastream like https://api.cosm.com/v2/feeds/61916/datastreams/random5.json, I'd expect to be able to page through all the datapoints in a feed like so:

GET ?start=1970-01-01T00:00:00Z&interval=0&offset=0&limit=1000
GET ?start=1970-01-01T00:00:00Z&interval=0&offset=1000&limit=1000
GET ?start=1970-01-01T00:00:00Z&interval=0&offset=2000&limit=1000
...

But there does not appear to be an offset parameter, and Cosm requires a start and an end parameter that are no further than 6h apart. Do I really need to issue thousands of requests (i.e. one or more for each 6h interval, starting from the beginning of time), or is there a better way?

errordeveloper
  • 6,716
  • 6
  • 41
  • 54
ejain
  • 3,456
  • 2
  • 34
  • 52

2 Answers2

0

At the moment, the way to paginate through every point is, as you say, to use interval=0, with the max per_page of 1000 and to split the request up into 6 hour chunks.

We're looking at increasing these limits now that our back end storage can handle it.

Sorry I can't give a better answer right now, but this should improve.

bjpirt
  • 236
  • 1
  • 1
  • Thanks! The 1000 per page limit seems reasonable; what's troublesome is the 6h interval limit. Consider the case where you have sporadic data going back several years; you'd have to issue thousands of requests, even if there were no more than a few hundred datapoints altogether! So if you could just go ahead and drop the interval limit requirement, that'd be great :-) – ejain Apr 25 '13 at 17:59
0

By setting the "interval" argument to the API, it seems to be possible to specify durations up to the maximum indicated by the documentation (up to an interval of 86400 seconds which supports a duration of up to a year).

This jsFiddle example shows a duration of approximately 3 weeks when used with an interval of 3600 seconds with a URL of the form:

http://api.cosm.com/v2/feeds/79903/datastreams/Temperature.json?start=2012-10-15T08:00:00Z&end=2012-11-08T00:00:00Z&interval=3600&limit=1000
James Kao
  • 1
  • Thanks, but this won't let me retrieve the original data points, just aggregated values. – ejain Apr 30 '13 at 07:44