11

I have this task that I'm undertaking where I would be reading data from a device and make it available over a web service. The data is read 4 times a second. I want the web clients to be have an open HTTP connection and get the device readings as a stream using chunked transfer as long as the client keeps the connection open.

As a proof of concept, I want to start with a service that constantly generates a random number, 4 times a second, wraps it in json and stream that to clients. I'm trying to model it loosely based on twitter streaming api.

I'm using restlet 2.1.2 to create that webservice but I'm not sure which Representation I should be using to achieve this. I tried searching for this but didn't find anything useful. Could someone point me in the right direction as to what I should be using and maybe some examples, perhaps.

Thanks

Professor Chaos
  • 8,850
  • 8
  • 38
  • 54
  • Do you mean what SubType of Representation should you use? JsonRepresentation should be the one: http://restlet.org/learn/javadocs/snapshot/jse/ext/org/restlet/ext/json/JsonRepresentation.html One of its constructors is handy: JsonRepresentation(Map map) – Diego Shevek Apr 18 '13 at 20:38
  • Are you bound to Restlet? I don't think it supports what you need out of the box without some [hacking](http://rfc2616.wordpress.com/2010/11/16/streaming-output-from-a-restlet-resource/). There's a web service library that handles async/steaming - [Atmosphere](https://github.com/Atmosphere/atmosphere). – pfyod Apr 19 '13 at 05:40
  • @DiegoAlcántara JsonRepresentation cannot be used to stream dynamic content, I don't think. – Professor Chaos Apr 19 '13 at 12:43
  • @pfyod Thanks for your response. unfortunately, I am bound to restlet. I did come across the hack you posted but did not have any luck gettin it to work with Thread.sleep() in the new Thread. – Professor Chaos Apr 19 '13 at 12:52
  • @ProfessorChaos what kind of client technology do you want/need to use to push and get data? F.ex. I don't think that you can consume twitter style streaming API with browsers' javascript+XHR. – pfyod Apr 19 '13 at 13:14
  • @pfyod The client right now is an android app. I plan to have a html5 style web app in the future. If I try [twitter api's url with some hashtag tracking](https://stream.twitter.com/1/statuses/filter.json?delimited=length&track=Google) it streams in the browser. It shows the json representation of the tweets as they come in. Browser shows loading but no content comes through when there are no tweets. I was wondering if that's possible do with restlet. – Professor Chaos Apr 19 '13 at 13:32
  • @pfyod [This post](http://stackoverflow.com/questions/16105665/restlet-streaming-with-thread-sleep) explains the difficulty I'm having with what I've tried so far. – Professor Chaos Apr 19 '13 at 13:37

1 Answers1

3

To achieve what you are trying to do, I'd use the WriterRepresentation (but see my answer to your other question), but I'm quite sure that you are going in the wrong architectural direction.

Indeed the following image from the documentation you linked

enter image description here

shows how even the Twitter streaming api is not intended to be connected by users, but by background processes that download messages in a store accessible by the HTTP. Users poll only the HTTP server, that reads the messages from the store and sends the back to the clients.

As a disconnected protocol, HTTP enable massive scalability that would not be possible otherwise. If each client establishes a persistent TCP connection backed by a dedicated server thread, you will rapidly exaust server resources! Moreover any HTTP proxy between the User Agent and the server could cause unexpected behaviours.

Thus, if you are bound to the HTTP protocol, the User Agent should poll. You can reduce the network load with headers like Last-Modified/If-Modified-Since or Etag/If-None-Match.

However, if you can adopt a different protocol, I strongly suggest to try a service bus over a connected TCP protocol.

Community
  • 1
  • 1
Giacomo Tesio
  • 7,144
  • 3
  • 31
  • 48
  • I have a very unique requirement and hence I'm down this path. The webservice will only every have 1 to 2 clients connecting, and just one for the most part so scalability is not a concern. – Professor Chaos Apr 21 '13 at 23:00
  • Still, HTTP proxies could become a problem. BTW, what's exactly the problem you are facing? In particular, why polling is not an option? – Giacomo Tesio Apr 21 '13 at 23:06
  • It's like,I am trying to create a server around an already existing client. The server that used to exist was written using a different technology and I do not have the source base to see how it was implemented. Polling was not used because the connection establishment and closing was considered an overhead but now this while loop is not proving to be any better but it is what it is and i'm trying to see if I can make the restlet service work. – Professor Chaos Apr 21 '13 at 23:18