How can I get the number of queued jobs of a particular type in gearman?

Question

I have a number of gearman clients sending a job, say job1.

$client = new GearmanClient();
$client->addServer();
$client->doBackground('job1', 'workload');

It takes, say 10 seconds to process this job. I want to track how many 'job1' jobs are waiting for a worker to work on them at any given time. How can I do that?

score 31 · Accepted Answer · edited Jun 14 '16 at 03:17

31

For quick checking, I use this bash one-liner:

(echo status ; sleep 0.1) | netcat 127.0.0.1 4730

This opens a connection to a gearman instance running on localhost, and sends the status query. This contains the name and number of jobs on that instance. The information can then be processed with grep/awk/wc etc. for reporting and alerting.

I also do the same with the workers query which shows all connected workers.

(echo workers ; sleep 0.1) | netcat 127.0.0.1 4730

The sleep is to keep the connection open long enough for the reply.

The full list of administrative commands, and what the output means is at http://gearman.org/protocol/. Just search for "Administrative Protocol".

edited Jun 14 '16 at 03:17

Nisse Engström

4,738
23
27
42

answered Jan 25 '11 at 13:19

d5ve

1,168
10
18

1

Just curious: why is the sleep 0.1 needed? The equivalent command for e.g. memcached doesn't need the sleep. – mjs Dec 02 '11 at 11:47
I had to use nc instead of netcat but otherwise worked great! – Nate Oct 24 '12 at 16:46

iandouglas · Answer 2 · 2016-08-23T03:59:27.330

12

To expand on d5ve's answer, add a -w parameter to "time out" your netcat connection, otherwise you never get back to a command prompt.

$ (echo status ; sleep 0.1) | sudo netcat 127.0.0.1 4730 -w 1

edited Aug 23 '16 at 03:59

answered Apr 29 '11 at 21:43

iandouglas

4,146
2
33
33

I can't propose a one character change, but using # as a prompt character makes the parser interpret the line of code as a comment, and dims it. – bschlueter Jun 14 '16 at 02:15

score 9 · Answer 3 · answered Jan 19 '16 at 04:02

9

telnet localhost 4730
status

worker_name total_queue currently_running number_of_workers
job1         1          1                 9

answered Jan 19 '16 at 04:02

Đọc truyện hay

1,913
21
17

1

Thanks for columns explanation – Aleksey Jun 19 '17 at 16:13
Double thanks on the columns! It doesn't seem to say anywhere – relipse Apr 06 '20 at 14:49

score 3 · Answer 4 · answered Mar 12 '18 at 16:41

I use gearman_top, which is part of mod-gearman.

Example output from the website:

+-----------------------+--------+-------+-------+---------+
| Name                  | Worker | Avail | Queue | Running |
+-----------------------+--------+-------+-------+---------+
| check_results         | 1      | 1     | 0     | 0       |
| host                  | 3      | 3     | 0     | 0       |
| service               | 3      | 3     | 0     | 0       |
| eventhandler          | 3      | 3     | 0     | 0       |
| servicegroup_jmx4perl | 3      | 3     | 0     | 0       |
| hostgroup_japan       | 3      | 3     | 0     | 0       |
+-----------------------+--------+-------+-------+---------+

score 1 · Answer 5 · answered Jul 31 '10 at 17:33

It doesn't look like there are any immediate ways to get this information.

Here are a few options. First, if you can grab job handles as you create them (search for "Speaking of checking the status"), you can store them away in some central place and query about them from any client.

Second, you can set your Gearman server to use persistent queues, and then run a query against the queue yourself. This might be the easier and cleaner of the two options.

+1 for using persistent queues: this also allows to have a better control on the queue — paul.ago, Nov 06 '14 at 11:56

score 1 · Answer 6 · answered Jan 20 '11 at 10:27

Gearmand has a telnet interface you can query. (the exact details of the protocol can be found on the gearman website - http://gearman.org/?id=protocol )

I used this code here as a starting point for rolling my own. https://github.com/liorbk/php/blob/master/GearmanTelnet.php (this code is perfectly good by itself and you should be able to drop use it out of the box)

It's a less than pretty solution but until someone improves gearman admin interface so you can talk directly via PHP or writes a plugin for it, you are on your own

score 1 · Answer 7 · answered Aug 06 '19 at 14:02

On Ubuntu 18.04 I have the gearadmin binary installed by default with the gearman package.

gearadmin --help
gearadmin --status can be used instead of the netcat alternative:

gearadmin --status
queue1    334     10      10

And a one liner:
while :; do gearadmin --status; sleep 1; done

How can I get the number of queued jobs of a particular type in gearman?

7 Answers7