Laravel Horizon not executing pending jobs - Kubernetes and Docker environment

Question

We have two different pods in Kubernetes for our Laravel app,

one running apache serving on port 80, (CMD /usr/sbin/apache2ctl -D FOREGROUND)
and another running worker (Laravel Horizon) (CMD php /var/www/artisan horizon)

The issue is when I check the horizon dashboard, it says 'Active', and I can see the Jobs in the 'Pending Jobs' section, but they never actually execute. They are just sitting there idle.

Now, when I SSH in the pod running apache and manually and run the command 'php artisan horizon' than it actually executes all pending jobs.

I have already ensured the followings:

Both the pods are connected with the same Redis database service
Horizon Prefix is the same for both the pods

I think you should run it with supervisor, install a supervisor in a container then executre the containers see what happens. — FULL STACK DEV, Jun 06 '20 at 19:19
See also https://stackoverflow.com/questions/47813800/issue-with-redis-managing-laravel-queues?noredirect=1&lq=1#comment121872506_47836681 — Ryan, Aug 27 '21 at 21:24

score 18 · Answer 1 · answered Apr 20 '21 at 07:37

18

Double check your APP_ENV matches one of the environments in the horizon.php config. Otherwise horizon will not start any queue workers.

By default only local and production environments are provided:

https://laravel.com/docs/8.x/horizon#environments

answered Apr 20 '21 at 07:37

Mostafa Bahri

2,303
2
19
26

1

In my case I had PRODUCTION instead of production - so remember it's case sensitive. – cby016 Sep 27 '21 at 17:48
Saved me hours of headaches. Thank you so much. – Jeremy Belolo Aug 15 '22 at 17:27

score 9 · Accepted Answer · answered Jun 07 '20 at 09:17

9

After struggling for days, I got the answer to this problem.

While using Redis as a cache, queue, or broadcast broker in the docker environment, we need to make sure that the following environment variables are defined properly and they must be the same across all the pods.

CACHE_PREFIX
REDIS_PREFIX
REDIS_CACHE_DB
HORIZON_PREFIX

Hope this will help others trying to deploy the Laravel apps using Kubernetes and Docker.

answered Jun 07 '20 at 09:17

Ambuj Soni

201
1
2
6

i'm running into the same issue. what type of redis are you using? we're trying an enterprise edition. doesn't seem to be working. anything else i should check. the env vars are exactly the same on both the API and the horizon "queue worker" pod – aibarra Jun 22 '20 at 01:29
@aibarra , I know its really late, hopefully, you might have figured it out yet. Just to answer you, I think the type of Redis doesn't matter, until or unless all your applications (all microservices) are talking with the same Redis instance. And if you already have the solution, it would be great if you can share it here, it can help other. Thanks! – Ambuj Soni Oct 01 '20 at 14:59
We were about to hit this issue and I stumbled to this post, gave a lot of heads up on the strategy, thanks – jision Dec 07 '20 at 14:32
Good Point Also In config/database.php We need to put Seperate REDIS_DB for CACHE, EVENT, SESSION and HORIZON – Rajnish Mishra Dec 17 '20 at 13:19
wow, thanks I deleted those from .env relying on the default config, now I know they must stay!!! – Daniel Katz Sep 14 '21 at 22:57
This was our issue as well. We had 2 Laravel apps on the same server, both using Redis (testing & staging of same application). This caused conflicts which in turn caused jobs to stay pending. We changed the config files though, not the .env entries e.g.: `'prefix' => env('HORIZON_PREFIX', Str::slug(env('APP_NAME', 'laravel'), '_').'_'.Str::slug(env('APP_ENV', 'unknown_env'), '_').'_horizon:'),`. Also we set REDIS_CACHE_DB to 2 on staging and REDIS_CACHE_DB to 3 on testing. – Ken May 31 '22 at 11:28
This solution was good starting point to troubleshoot the issue further. For me just keeping `APP_NAME` unique for each service solved the issue. – Jigar Feb 15 '23 at 12:54

score 5 · Answer 3 · answered Mar 18 '21 at 20:04

5

In my case, I need to change my app environment from prod to production

APP_ENV=production

answered Mar 18 '21 at 20:04

Linh Pham

206
4
2

In my case I had PRODUCTION instead of production - so remember it's case sensitive. – cby016 Sep 27 '21 at 17:48

score 1 · Answer 4 · answered Sep 14 '21 at 23:18

1

In my case I added the jobs into "emails" queue, but horizon.php config file didn't specify this queue name for supervisor-1 :-)

answered Sep 14 '21 at 23:18

Daniel Katz

2,271
3
25
27

score 0 · Answer 5 · answered Jul 17 '22 at 11:01

0

I just restart Redis server

/etc/init.d/redis-server restart

answered Jul 17 '22 at 11:01

İlqar Huseynli

1

score 0 · Answer 6 · answered Jan 22 '23 at 09:02

If jobs or listeners send requests to external services and cannot reach destination hosts and the connection timeout value is set to something very big or job timeout value is also set to a big value then the jobs might also be in pending state long enough so that it may seem that horizon is not executing them.

Laravel Horizon not executing pending jobs - Kubernetes and Docker environment

6 Answers6