Laravel queue rate limiting not executing work

Question

I'm attempting to implement Laravel 5.7's queue job rate limiting which for use when queue jobs hit an external API that's rate limited.

Here's my job:

    public function handle() {
        echo 'about to check throttling'.PHP_EOL;
        Redis::throttle('throttle-test')->allow(10)->every(5)->then(function () {
            // this is never executed
            echo 'doing work'.PHP_EOL;
        }, function () {
            // also never executed
            echo 'released back onto queue'.PHP_EOL;
            return $this->release(10);
        });
    }

There's no mention of needing to use Redis for queues, cache or anything of that nature in the docs, aside from "...application can interact with a Redis server." Either way, here's my env vars:

DB_CONNECTION=mysql
CACHE_DRIVER=file
SESSION_DRIVER=file
QUEUE_DRIVER=sync
REDIS_HOST=redis

I've confirmed the RedisServiceProvider is receiving the correct configuration it expects:

array:3 [
  "client" => "predis"
  "default" => array:4 [
    "host" => "redis"
    "password" => null
    "port" => "6379"
    "database" => 0
  ]
  "horizon" => array:5 [
    "host" => "redis"
    "password" => null
    "port" => "6379"
    "database" => 0
    "options" => array:1 [
      "prefix" => "horizon:"
    ]
  ]
]

I'm struggling to understand why there's no runtime errors, it's just that nothings get executed. If I comment out the throttle stuff the job runs fine and does what it's supposed to do. What am I missing about the requirements to use this throttling?

Check `storage/logs/laravel.log` last lines, Queue generally failed silently but written to Log when errors occured. — KeitelDOG, Mar 01 '19 at 02:30
Check `storage/loogs/laravel.log` last lines, Queue generally failed silently but written to Log when errors occured. — KeitelDOG, Mar 01 '19 at 02:30
The main issue I always have with anything queue related in Laravel is that debugging is a pain. You can always `Log::info()` rather than `echo` but generally checking the logfile is the only way to find out what's going on. — Joe, Mar 01 '19 at 09:05
With my queue driver being "sync" it's all being executed in the current process and not queued so an `echo` should be sufficient. My logs go to stdout anyways and I'm not seeing anything there. — Ben, Mar 01 '19 at 12:35
In Laravel Docs it says that `Rate Limiting. This feature requires that your application can interact with a Redis server.` So since you're using `QUEUE_DRIVER=sync`, your queue never get handled by Redis, so it would never be executed. Rate Limiting is a REDIS-ONLY feature. — KeitelDOG, Mar 03 '19 at 20:57
`interact with a Redis server` and `using redis as your queue` are 2 different things. I think it would specify a queue if it required Redis being used as a queue. I've confirmed the Redis store information is correctly being passed to the `RedisServiceProvider` — Ben, Mar 04 '19 at 01:47
I'm deployed using Docker, queues are managed via Laravel Horizon. — Ben, Mar 05 '19 at 12:31
Does `redis` actually resolve through DNS to your redis host? I'm not completely familiar with how docker handles internal networking in this regard, but including some of your Dockerfile might help. Also, I know it's been mentioned already but would you try changing your `echo` statements to `logger()->debug('starting/working/released')`? Even if using the sync driver *should* be enough for echo statements to appear, it would just help us completely rule out that possibility. — Travis Britz, Mar 09 '19 at 19:38
Yes, it does resolve as other elements with my app (such as caching) rely on it. — Ben, Mar 09 '19 at 23:34
Have you attempted to use `redis` as the queue driver in your `.env` file? — iLC, Mar 12 '19 at 01:00

score 6 · Accepted Answer · answered Mar 12 '19 at 01:26

6

The documentation for Laravel in this case is very misleading. While it implies you only need a connection to Redis you actually have to be using Redis for your queues.

answered Mar 12 '19 at 01:26

Ben

60,438
111
314
488

No! You don't need to use redis for your queue if you use the rate limiters. This answer is not correct! – Mohamed Said Jun 19 '20 at 04:59

score 0 · Answer 2 · answered Aug 02 '19 at 15:40

Unfortunately there is no "official" way to rate limit queue without using Redis queue driver.

I wrote custom queue worker mxl/laravel-queue-rate-limit that uses Illuminate\Cache\RateLimiter to rate limit job execution (the same one that used internally by Laravel to rate limit HTTP requests).

Read more about its usage on GitHub page or in this SO answer.

Laravel queue rate limiting not executing work

2 Answers2