2

I have a rails application that is serving as a backend API and Indexer that has to display the most up to date information for a number of different products. So, and this is not by my design, the only way that we can get updates from the current system is by receiving a number of XML files at random and numerous points throughout the day, which I need to scrape and put into my DB.

So my question is two fold and has to do with how I can execute such a job.

First, I have looked around at various cron executors in Ruby and seen that the whenever gem is a popular choice for basic scheduled cron tasks, but since this isn't a scheduled cron job I was wondering how I could prompt such an event and what tools I could use to accomplish this.

Second, I was wondering if it was at all possible to trigger the cron jobs from my first question whenever a directory in the Rails App was updated? I.E. whenever I get a new XML file in that directory I execute my script to scrape it and put it in the DB.

Thanks!

Zubatman
  • 1,235
  • 3
  • 17
  • 34
  • What is your latency requirement (or put another way, what is the longest time gap between new file arriving and getting processed)? Even if this is as little as five minutes or perhaps less, I would use `whenever`. Triggering off files arriving sounds like an approach which could lead to race conditions. – steve klein Jul 27 '15 at 21:11

1 Answers1

1

First: cron is a Unix utility which, when set up on your system, can periodically run tasks (called "cron jobs") like you're looking for. However, since it's a Unix utility and in no way connected to Rails or Ruby, it would need to be installed directly on your server and configured outside of your Rails app. The exact details for that depend on your server's environment.

Once you have cron installed and running, if you wanted to execute some command in your app once per hour, you would add an entry to your /etc/crontab (the real path depends on how you installed it) file like:

0 * * * * /home/yourapp/bin/rails runner -e production 'SomeCommand.run'

Now, the whenever gem can help you automatically generate your cron file, but it won't actually run anything unless you copy the output of whenever to your crontab. There are some pure-Ruby cron alternatives, like rufus-scheduler, which you might prefer. You'll still need to have the script which runs it be a long-running script, though, which will require some server configuration (most importantly, if it dies for any reason, you'll want a process monitor like god to restart it for you.

Second: Yes, that's possible! watching a directory in ruby has some details for you. You'll still need to set up a long-running script to be watching that folder, and execute your hook when a file is added.

Community
  • 1
  • 1
Robert Nubel
  • 7,104
  • 1
  • 18
  • 30