I am new to Apache Storm. I have currently done word-count problem. Currently my data source (file containing words) is in my local system. But now i need this file to be in external system. i.e. if i tell ip of external system and path to file, then is there way that my application will get data from that file (remotely located)?
Asked
Active
Viewed 152 times
0
-
One typical approach is to use a distributed queue, e.g., kafka. You can stream your file onto the queue and have storm read from the queue. – lorcan Oct 29 '13 at 15:12
-
@lorcan, why don't you put this as an answer? – SSaikia_JtheRocker Oct 29 '13 at 18:28
-
may take a look [here](http://stackoverflow.com/questions/19510252/how-to-make-my-storm-topology-to-work-real-time/19511569#19511569) – user2720864 Oct 29 '13 at 20:14
1 Answers
0
One option is to feed your file to MongoDB and let your spout pull the file from MongoDB. Have a look at Storm Mongo. But please keep in mind the size of your file. You don't want to transfer huge files as 'tuples'.
Chiron
- 20,081
- 17
- 81
- 133