0

We are trying to build a real time ticker app with financial data being pushed from server to client at the rate of 7 Million records per day. Clients could be browser based webapp or Native/Phonegap Hybrid mobile app.

Option 1: a part of my team is doing a PoC using the Long Polling approach with the help of Asynchronous server (at server side) and some jScript+Ajax (at client side).

Option 2: while am still reading about different techniques, am wondering why not use socketIO + Node JS. They will take care of any fallback technique for unsupported clients and do better at Scaling and Resiliency aspects.

Is there any reason why Option 1 might be best ? or is there any Option 3 that we can think about ?

Note: We have a limitation on Websockets, our Websphere 8.5.5.4 (non liberty profile) does not support websockets.

yathirigan
  • 5,619
  • 22
  • 66
  • 104
  • Did you search about xmpp, ejabberd || openfire http://xmpp.org/ & https://www.ejabberd.im/ – B.Kocaman Jun 28 '16 at 15:50
  • that is a new one am hearing now. how different or advantageous it is when compared to websockets ? – yathirigan Jun 28 '16 at 16:07
  • Actually about my bad English I won't get explain but there is also some good explanations; :) http://stackoverflow.com/questions/26549010/xmpp-vs-websocket | http://stackoverflow.com/questions/11994061/using-xmpp-or-websocket-why-there-is-a-server-needed-in-real-time-communication | https://en.wikipedia.org/wiki/XMPP – B.Kocaman Jun 28 '16 at 16:18
  • i just read this http://stackoverflow.com/questions/26549010/xmpp-vs-websocket and looks like xmpp mainly helps to build a chatty application where there are peer-to-peer conversations and much more chat application requirements. Ours will be more of a financial data ticker app where most of the times, data will come from the server to clients only – yathirigan Jun 28 '16 at 16:24
  • We are using xmpp for push notification & chat for our android applications. Before decide to use xmpp, we tried GCM, socket.io + elephant.io – B.Kocaman Jun 28 '16 at 17:05
  • if an app doesn't not have Chat kind of requirement (client does not send data to server), i believe xmpp might be an over kill – yathirigan Jun 28 '16 at 17:19
  • 1
    As best I know, webSocket is the most efficient way to regular push data from server to client. HTTP long polling has more overhead that one continuous webSocket. – jfriend00 Jun 28 '16 at 21:39
  • 1
    7 million records per day to a single client is a lot of records. That's 81 records per second if you're doing them equally spread throughout the 24 hours and a lot worse than that if most comes during business hours. If you're really sending that many to each client you are going to have to batch them into chunks to get any sort of decent throughput (this is independent of polling vs. websocket). If you have lots of connected clients too, then multiply it even more. – jfriend00 Jun 28 '16 at 21:40
  • This seems like you need to first do some sanity checks about how much data you're sending and how much raw throughput you think you will have. I also don't see how a mobile app could possibly be the recipient of this much data and have any chance at maintaining battery life. – jfriend00 Jun 28 '16 at 21:41
  • who can read 7 millions of records a day ?! even jfriend00 can't do this ^^ . Even for some sort of analytics with front end based computation but still (in my opinion ) this is a bad design – Revln9 Jun 28 '16 at 22:52
  • 1
    One of the most used solutions for pushing real-time market data is Lightstreamer (full disclosure: I work for Lightstreamer). Basically, you will offload all the real-time traffic to Lightstreamer Server, keeping Websphere for all the request/respinse traffic. Check out some live demos: http://demos.lightstreamer.com/ – Alessandro Alinone Jun 30 '16 at 08:57
  • the client application will receive data only when they user actually logs into the system. User could log only for 2 or 3 hours only per day and that particular user would receive data feed only for that specific duration. We are still trying to understand the user behavior and data rate. As of now, we are thinking about SocketIO + NodeJS , but yet to finalize – yathirigan Jul 01 '16 at 11:01

0 Answers0