(Note: I am a beginner in using AWS SQS for queues) I have a function where I would like to insert tens of thousands records into an excel, save the excel to AWS S3 and display into frontend datatable. This function executes using AWS SQS queue with Supervisor as worker in a Laravel 9 webapp.
The error that I am getting is:
- Job\SomeJob has been attempted too many times or run too long. The job may have previously timed out. {"exception":"[object] (Illuminate\Queue\MaxAttemptsExceededException(code: 0)
- (Symfony\Component\ErrorHandler\Error\FatalError(code: 0): Maximum execution time of 60 seconds exceeded at /var/app/current/vendor/laravel/framework/src/Illuminate/Collections/Arr.php:314)
I have no clue why I am getting this error but the job is actually successful. This error will show up in the failed_jobs table and I have a function where if there are any failed_jobs, run a script to email to the manager and I believe you guys know what happens after that.
What I have tried is to Log::info() every line before and after a process to know which is causing the error.
My Supervisor setting for SQS:
[program:sqs-worker]
process_name=%(program_name)s_%(process_num)02d
command=php /var/app/current/artisan queue:work sqs --sleep=3 --tries=1 --timeout=1800
autostart=true
autorestart=true
user=webapp
numprocs=1
redirect_stderr=true
stdout_logfile=/var/www/html/worker.log
How I dispatch the job:
class SomeOtherController extends Controller{
public function show(){
dispatch(new SomeJob($id));
return 'job run';
}
}
The job content is:
Class SomeJob implements ShouldQueue{
use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
public $timeout = 1800;
public $id;
public function __construct($id){
$this->id = $id;
}
public function handle(){
Log::info('start job');
$apps = Application::where('client_id', $id)->get(); //15000+ records
Log::info('start foreach');
$count = 0; //to count each 100 records, do Log::info()
foreach($apps as $key => $app){
if($count == 100){
Log::info('This is the '.$key.' record');
$count = 0;
}
//the actual job is much more lenghty and complicated
$list = new ApplicationTable();
$list->client_id = $app->client_id;
$list->name = $app->name;
$list->booking = $app->booking->name;
$list->price = $app->price + $app->discount + $app->gst;
$list->save();
$count++;
}
Log::info('end foreach');
//Some process to generate and store excel to S3
$lists = ApplicationTable::where('client_id','=',$id)->get();
(new ReportExport($lists))->store('application_report');
$s3 = Storage::disk('s3');
$s3_path = 'ApplicationReport';
$s3->put($s3_path, file_get_contents('application_report'));
//unlink
unlink($path);
$user_email = $apps->first()->user->email;
if(isset($user_email)){
\Mail::to($user_email)->send(new ApplicationReportMail($id));
}
log::info('end job');
return true;
}
}
What I am expecting is that the log will show all processes and ends with 'end job' without any error. But what I am getting is:
[20XX-XX-XX 12:56:34] start job
[20XX-XX-XX 12:56:36] start foreach
[20XX-XX-XX 12:56:41] This is the 100 record
[20XX-XX-XX 12:56:47] This is the 200 record
[20XX-XX-XX 12:56:52] This is the 300 record
[20XX-XX-XX 12:56:57] This is the 400 record
[20XX-XX-XX 12:57:04] local.ERROR: App\Jobs\SomeJob has been attempted too many times or run too long. The job may have previously timed out. {"exception":"[object] (Illuminate\\Queue\\MaxAttemptsExceededException(code: 0): App\\Jobs\\SomeJob has been attempted too many times or run too long. The job may have previously timed out. at /var/app/current/vendor/laravel/framework/src/Illuminate/Queue/Worker.php:746)"
[20XX-XX-XX 12:57:06] This is the 500 record
[20XX-XX-XX 12:57:10] This is the 600 record
...
[20XX-XX-XX 13:09:46] This is the 11400 record
[20XX-XX-XX 13:09:52] This is the 11500 record
[20XX-XX-XX 13:09:53] Maximum execution time of 60 seconds exceeded {"userId":144,"exception":"[object] (Symfony\\Component\\ErrorHandler\\Error\\FatalError(code: 0): Maximum execution time of 60 seconds exceeded at /var/app/current/vendor/laravel/framework/src/Illuminate/Collections/Arr.php:314)"
[20XX-XX-XX 13:16:20] local.INFO: end foreach
[20XX-XX-XX 13:16:23] local.INFO: end job
As you can see from the logs, the job was running and after roughly 30-60 seconds, Laravel throws the MaxAttemptsExceededException exception. Then at 13:09:53 get another FatalError exception where it says 60 seconds timeout exceeded and the log stopped. It continues after 13:16:20 to finish the process...
For anyone curious on what the config for the queue inside the failed_jobs table:
...,"maxTries":null,"maxExceptions":null,"failOnTimeout":false,"backoff":null,"timeout":1800,"retryUntil":null,...
Really appreciate any input and clarification on this matter. I have search for a solution but to no success.
