1

Currently we are using apache flume framework to collect the logs from all our web applications and it will store the logs in MongoDB database table called “Raw_Data”. Now we got a task to differentiate the logs and store the logging information in different tables based on the application. So we decided to maintain a separate table for each application. Is there any way to create and run stored JavaScript on a frequency like scheduler to load the data application wise in MongoDB?

I’m very much familiar with oracle DB but I’m new to MongoDb. Can someone help me to find out the required things for this asap.

Thanks Kishore

Jai Kishore
  • 77
  • 1
  • 2
  • 7
  • If you want it to be JavaScript, write some code for NodeJS as JavaScript and call it using chron for example: `node myscheduledtask.js`. – WiredPrairie May 30 '13 at 21:28
  • Thanks for the information. After looking into your solution, I have below queries. Can we insert data into a different table thru Stored java script functions if so can you provide some links for this? Can you refer some sites for scheduling mongo scripts with Python? – Jai Kishore May 30 '13 at 22:29
  • (You just pasted the same comment?) I said use NodeJS, not Python. Just write it using NodeJS as I suggested originally in my comment. You tagged this as "mongoose" which is only for NodeJS. Please do a little more research on your own. – WiredPrairie May 31 '13 at 00:34

1 Answers1

2

MongoDB has stored Javascript procedures, but no scheduler.

http://docs.mongodb.org/manual/tutorial/store-javascript-function-on-server/

You could trigger this from a client machine using a cron job and the mongo shell. However, I strongly recommend simply running a script in Python or similar client-side scripting language. It'll be far easier to version-control and to debug than server-side Javascript.

A. Jesse Jiryu Davis
  • 23,641
  • 4
  • 57
  • 70
  • 1
    First line of the docs: "Note We do not recommend using server-side stored functions if possible." MongoDB doesn't really have stored procedures. – WiredPrairie May 30 '13 at 21:24
  • Jesse, Thanks for the information. After looking into your solution, I have below queries. Can we insert data into a different table thru Stored java script functions if so can you provide some links for this? Can you refer some sites for scheduling mongo scripts with Python? – Jai Kishore May 30 '13 at 22:28
  • Google for "crontab" and for "pymongo". – A. Jesse Jiryu Davis Jun 03 '13 at 15:56