0

I'm building a testing utility, which runs a bunch of tests periodically, some scheduled to run daily, some hourly, some weekly, etc. I'm trying to build this feature on Celery Beat, with a RabbitMQ broker. However, the list of tests to be run at every such period can change (tests can be added, edited, or deleted). I need to schedule the tests in this dynamic list of tests.

This is what I have so far:

def test_list():
    tests = ['t1','t2','t4','t5','t7']
    return tests

app.conf.update(
    CELERYBEAT_SCHEDULE={
        'schedule_task':{
            'task':'test_celery.tasks.test_scheduler',
            'schedule': timedelta(seconds=10),
            # 'args': (test_list(),)
        }
    }
)

@app.task
def test_scheduler():
    tests = test_list()
    for test in tests:
        print "RUNNING TEST {}".format(test)

I've tried passing the list of tests to be run - (a) as a parameter to 'args' of the CELERYBEAT_SCHEDULE config, (b) as a function call returning the list of tests to the args parameter of the CELERYBEAT_SCHEDULE config, and (c) by calling the test_list() function in the test_scheduler() function. None of these three techniques seem to be able to handle a list that can be dynamically updated (For instance, if I add another test - t8 - to the list of tests, ideally, on the next scheduled run of the test_schedule() function, I would want "RUNNING TEST t8" printed on the console, without restarting Celery Beat.)

Please note that the scheduler cannot be stopped/restarted, it must be running all the time.

Any guidance is appreciated, thanks in advance!

sukrit6
  • 11
  • 1
  • 3
  • Seems to be a duplicate: http://stackoverflow.com/questions/10194975/how-to-dynamically-add-remove-periodic-tasks-to-celery-celerybeat – skovorodkin Oct 03 '16 at 15:23
  • @skovorodkin, I forgot to mention, like the OP in that post, I am looking for a solution which does not use Django/djcelery. – sukrit6 Oct 04 '16 at 05:53

0 Answers0