0

I'm using Laravel 5.2. I currently have a cron job where it fetches many thousands of rows from my database and performs computations on each of those rows. The cron job currently takes roughly an hour to run, but the database continues to get larger every day.

One option would be to manually add more cron jobs using crontab -e (i.e. 5 cron jobs that split the job into 5 smaller parts). However, is it also possible to do something like the following:

  1. Set up a single cron job using crontab -e.
  2. The cron job script gets all the rows from the database and, depending on how many rows are returned, it determines how many cron jobs it should further be split into. For example, if the database returns 1000 rows, it should split it into 10 cron jobs, but if the database returns 10000 rows, it should split it into 100 cron jobs.

So my question is, is it possible to have only a single cron job listed in crontab -e, where the PHP script then decides how many more cron jobs it should be split into (essentially calling more cron jobs from a PHP script)?

Is it safe to do this? Is there a better way to approach this?

Here is my current script:

class PostCron extends Command
{
    /**
     * The name and signature of the console command.
     *
     * @var string
     */
    protected $signature = 'post_cron';

    /**
     * The console command description.
     *
     * @var string
     */
    protected $description = '';

    /**
     * Execute the console command.
     *
     * @return mixed
     */
    public function handle()
    {
        $posts = Post::all(); // currently returns ~1000 rows, but grows in size daily

        foreach ($posts as $post) {
            // computations on each separate post
        }
    }
}

2 Answers2

0

I would suggest to use queueing. There are tools to do that, like Beanstalkd. Then you have one cronjob which adds several tasks to the queue.

Dieter Pollier
  • 689
  • 1
  • 5
  • 11
0

you can create job and run each post into queue. refer here laravel queue

so your code will be something like below

protected $bus;
/**
 * Create a new command instance.
 *
 * @return void
 */
public function __construct(\Illuminate\Contracts\Bus\Dispatcher $bus)
{
    parent::__construct();

    $this->bus = $bus;
}

public function handle()
{
   $posts = Post::all(); // currently returns ~1000 rows, but grows in size daily

   foreach ($posts as $post) {
       // computations on each separate post
       $this->bus->dispatch(new YourJobClassName($post));
   }
}
xmhafiz
  • 3,482
  • 1
  • 18
  • 26
  • The computations I'm doing benefit more from a cron job, and not a queue. They NEED to be run every hour and not added into a queue in the hopes that it will finish within an hour. –  Jun 07 '16 at 05:14
  • yes, your cron will execute the computation jobs in queue. you also can set number of process for queue to run simultaneously to make it fast https://laravel.com/docs/5.2/queues#supervisor-configuration. – xmhafiz Jun 07 '16 at 06:03
  • similar problem here http://stackoverflow.com/questions/35906749/laravel-run-multiple-scheduled-tasks?rq=1 – xmhafiz Jun 07 '16 at 06:05
  • What I'm saying is that I WANT to use cron jobs, not queues. –  Jun 07 '16 at 06:20
  • depends, i think the best case is using cron with queue. but if you want to run multiple crons, maybe can add same command in your Kernel.php but surely not the good option – xmhafiz Jun 07 '16 at 07:01