1

I have an uploader service which needs to run every 5minutes and it definitely finished within 5 minutes so there are never two parallel session.

Wondering what would be a good strategy to run this, either to schedule this as a cron job on host or start a go program with infinite loop which execute the program and sleeps(Golang: Implementing a cron / executing tasks at a specific time)

rohit
  • 862
  • 12
  • 26
  • 3
    The cron job can eliminate the need to monitor and restart the long running process. –  Nov 14 '19 at 01:50
  • @iLoveReflection so could a process manager, like upstart or systemd. – Adrian Nov 14 '19 at 14:39
  • If you don't care what specific time it runs, just that it runs every five minutes, the linked solution is way overkill. You could just use a [Ticker](https://golang.org/pkg/time/#Tick) and be done with it in a couple lines of code. – Adrian Nov 14 '19 at 14:42

2 Answers2

3

If your task is...

  • On Unix
  • Stand alone
  • Periodic
  • Has an acceptable startup time

cron will be better than rolling your own scheduler just for the one service. It will guarantee the process will always run at the correct time and has rudimentary error reporting. There's no need to add a watchdog in case your infinite loop has an error, cron will run the process again in 5 minutes.

If cron is insufficient, look into other job schedulers before rolling your own.

I have an uploader service which needs to run every 5minutes and it definitely finished within 5 minutes so there are never two parallel session.

These are famous last words. I would suggest adding in some form of locking. For example, write your PID to a file in /var/run and check if that process is running. There's even a little pidfile library for Go.

Schwern
  • 153,029
  • 25
  • 195
  • 336
  • "rolling your own scheduler just for the one service" makes it sound like way more work than it is. A repeating task every 5 minutes is two or three lines of code in Go. See https://golang.org/pkg/time/#Tick. – Adrian Nov 14 '19 at 14:42
  • 1
    @Adrian What if the process dies? Or leaks memory? Or the system restarts? Or if it needs to be run at different periods? Or only on weekdays? – Schwern Nov 17 '19 at 07:54
  • In order: process manager; anything can have a bug; process manager; that would be outside the scope of what was asked; that would also be outside the scope of what was asked. And all of that seems unrelated to my point - your answer implied it would be somehow burdensome to write a Go program that does something every five minutes (the thing that was asked) when it is in fact trivial. – Adrian Nov 18 '19 at 14:13
  • 1
    @Adrian The question is about an infinite loop vs a job scheduler. It takes a lot more than Tick to replicate the features and reliability of a proper job scheduler. If you feel we've got it wrong, please write up your answer. – Schwern Nov 18 '19 at 17:52
  • 1
    @Adrian Consider that the goal is to *reliably and automatically* run a process every 5 minutes. Manually checking a process manager to see if it hung or died or restarted is neither. Exactly *because* anything can have a bug the process needs to take exceptional cases into account. A good job scheduler and PID lock file will reliably run the process every 5 minutes whether it hung, died, or the system restarted. And it will handle overlaps. AFAIK Ticker won't do that. I could be wrong, Go is pretty awesome, so I look forward to your answer. – Schwern Nov 18 '19 at 19:15
  • "Manually checking a process manager" do what now? The point of a process manager is to do that automatically. If the process manager itself hangs, you've got bigger problems, like your entire system going to pieces. Any mainstream process manager (systemd, upstart, runit, etc) is going to be pretty reliable. You might as well say "don't use cron because you'll just have to manually check that cron doesn't hang or die". – Adrian Nov 18 '19 at 19:41
  • @Adrian I must not be understanding your plan. You should post an answer. – Schwern Nov 19 '19 at 02:05
0

Take a look on Systemd, you can execute a script with timers and set max execution time for the script.

https://wiki.archlinux.org/index.php/Systemd/Timers

Labradorcode
  • 381
  • 3
  • 15