Starling: How to speed up the queue

Question

I've launched a website today which makes heavy use of Starling and Workling to handle the background task queue.

Traffic has been heavy and although my server CPU/memory has not been under too much strain, the job queue is seriously backing up.

I am inexperienced with both Starling and Workling and urgently need to understand how I can speed up the queue.

This is probably quite a simple configuration issue but all suggestions and thoughts would be very appreciated.

Please let me know if you need more information.

score 1 · Answer 1 · answered Mar 30 '09 at 11:23

1

I worked out how to do this in the end, it's very simple. Just start up more workling processes from the command line - there can be as many running as there is memory for. They will all pick up jobs from one or more instances of Starling, as specified in workling.yml. Note that :multiple must be => true in script/workling_client for this to work.

answered Mar 30 '09 at 11:23

netflux

3,648
6
37
40

May sound stupid, but is nice to check if you worklings are using the Workling::Remote::Invokers::ThreadedPoller. This way, each one of your workers will have his own thread inside the workling process. – lsdr Sep 28 '09 at 17:06

score 0 · Accepted Answer · answered Mar 25 '09 at 02:46

If you have access to another server you can split the tasks up between the servers. You've probably seen it but Railscasts has a pretty good introduction to Starling and Workling.

There is also a book you can view on Google books with more advanced information:

Practical Rails Plugins

Starling: How to speed up the queue

2 Answers2