0

I have an R model, which predicts user ratings and writes them to database. The question is - what is an optimal way to update these predictions using new incoming data? Now I have a cron task that updates ratings every night, but we need more frequent updates.

What is the proper solution in this situation?

I'm thinking of running a 'continuous' program that connects to database, grabs new data, updates predictions, sets timeout and repeats. I like this solution more than launching cron task because I want to save time loading 1Gb-workspace into memory. Is it ok, what are the major drawbacks?

I understand this is a very dirty solution, we will definitely follow one of the advice here , but now we need a temporary solution

Community
  • 1
  • 1
RInatM
  • 1,208
  • 1
  • 17
  • 39
  • I think I had a similar situation a while back. See if http://stackoverflow.com/questions/4780632/monitoring-for-changes-in-files-in-real-time gives any ideas. – Roman Luštrik Mar 27 '15 at 08:34
  • What do you mean by "grabs new data and updates predictions". Does that mean re-training the model and doing a new round of predictions based on the updated model? – LauriK Mar 27 '15 at 08:39
  • no, I don't update the model, just make new predictions depending on new data – RInatM Mar 27 '15 at 12:16
  • @RomanLuštrik thanks for the link! in my case the data is updated in database (1000 new events per hour) so there is no need for the monitor. I also hope there are any changes since 2011 – RInatM Mar 27 '15 at 13:36

0 Answers0