AWS Elastic Beanstalk - Starting SWF Background Workers

Question

I have been trying to find out the best way to run background jobs using PHP on AWS Elastic beanstalk, and after many hours searching on Google and SO, I believe that one good solution is using SWF and activity workers.

I found this example buried in the aws-sdk-for-php: https://github.com/amazonwebservices/aws-sdk-for-php/tree/master/_samples/AmazonSimpleWorkflow/cron

The read-me file says:

To run this sample, you need to execute three scripts from the command line in separate terminal/console windows

and

Note that the start_cron_example_workflow.php script will exit quickly while the decider and activity worker scripts keep running until you manually terminate them.

the decider and activity worker will loop "forever", and trying to run these in EB is what I'm having trouble doing.

In my .ebextensions directory I have a file that executes these files:

container_commands:
  01background_task: 
    command: "php -f start_cron_example_activity_workers.php"
  02background_task: 
    command: "php -f start_cron_example_workflow_workers.php"

But I get the following error messages:

ERROR
Failed to deploy application version.

ERROR Some instances have not responded to commands. Responses were not received from [i-a5417ed4].

Any way I can do this using config files? How can I make this work in AWS EB without introducing a single point of failure?

Thank you.

Answer 1

You might consider using a service like IronWorker — this is specifically designed for what you are trying to do and will probably work better than putting together your own solution on a micro instance.

I have not used Iron.io yet, but was evaluating it as I am looking to move my stuff over to AWS so I need to have cron jobs handled as well.

Answer 2

Have you taken a look at the Fat Controller ? It can daemonise anything. There's documentation and examples on the website: http://fat-controller.sourceforge.net