6

I'm actively looking for a real time big data analysis solution.

for now I've identified a mature actor Twitter Storm and also a young challenger Spring XD.

Does anyone know if Spring XD could real time process log event streams or twitter streams as well as Storm would do ?

What are the know limitations of both technologies ?

tom
  • 1,647
  • 14
  • 15

2 Answers2

4

SpringXD is a great tool if you are looking to string together several modules without having to write too much code. We used it for a little bit in production and found that it let us scale.

SpringXD is worth using if you don't have to create too many custom modules, need containers for stability, and don't have too elastic of a load.

SpringXD also allows quick integration with several technologies (Redis, RabbitMQ, Hadoop to name a few) out of the box. If you are looking to get something quickly distributed, it might be a great tool for your use case. However, if you have a very elastic load, or want to deploy several different streams, then they don't currently recommend that.

poy
  • 10,063
  • 9
  • 49
  • 74
  • "they don't currently recommend that" - do you have a reference to that? thanks. – spakendralo man Jul 08 '15 at 09:08
  • @spakendraloman This came from working with directly. This might have also been improved sense I last worked with them (beginning of the year). – poy Jul 08 '15 at 20:48
0

I am interested in the same thing, I've worked with Spring for years, and recently Storm. my guess is that they both will perform very well..I am using Storm currently simply because it's more mature..However, if it was a Spring Based project..i would recommend giving Spring XD a shot before Storm.

Jeryl Cook
  • 989
  • 17
  • 40
  • I'm interested in taking notes from this discussion, too bad that no one seems to have offered anything of value as yet. I'm looking to build a platform for an IoT system that can manage data streams from hundreds (or thousands) of devices, perform near real time analytics, real time scoring using predictive models and visualization at the application level. I've narrowed my options down to YARN + Spark + Spark Streaming + MLBase + R + python + Spark XD (maybe) + JMS. This list of components is growing and I'm not quite a fan of that. Anyone have experience in setting up something similar? – Arun Jose Apr 24 '14 at 05:22