0

Every day a PHP script is executed by cron job to send emails to subscribers.

The script works fine, But the issue is that the subscribers are increased and The script is timing out after 100 sec.

So I need to run process in batches.

My question: How to let the script to start from a certain id, then the next time start from this id? I cannot divide the script into different files as I don't know how many subscribers there are. 

/* Get all subscribers from DB */
$stmt = $pdo->query("SELECT * FROM subscribers");

/* Loop through subscribers and get email */
foreach(){

    /* Create a new PHPMailer object. */
    $mail = new PHPMailer();

    /* Set the mail sender. */
    $mail->setFrom('myemail@domain.com', 'John Doe');

    /* Add a recipient. */
    $mail->addAddress('recipient@recipient.com', 'recipient');

    /* Set the subject. */
    $mail->Subject = 'New article';

    /* Set the mail message body. */
    $mail->Body = 'This is a new article.';

    /* Finally send the mail. */
    if (!$mail->send())
    {
       /* PHPMailer error. */
       echo $mail->ErrorInfo;
    }

}

UPDATE:

Would one of these methods work?

1- Instead of fetching all the subscribers at once SELECT * FROM subscribers , I use a loop to get 500 rows per time and maybe use sleep() after each time.

/* Loop with increment of 500 */
for($i = 0; $i <= $rowsCount; $i += 500){

    /* Select 500 subscribers */
    $query = ("SELECT * FROM `subscribers` LIMIT $i, 500");

    /* Send Emails to 500 subscribers */

    sleep(60);

}

2- Save the last subscriber id in a table and each time the script is executed, Start from that id:

/* Get the last subscriber id from previous time */
$last_subscriber_id = `SELECT id FROM last_subscriber_id`;

/* Select the next 500 subscribers starting from the previous time last id */
$query = "SELECT * FROM `subscribers` WHERE `id` > $last_subscriber_id LIMIT 500";

/* Send Emails to 500 subscribers */
..

/* Update  last_subscriber_id with the last id */
..

But in this case I would run the script every x minutes, As I don't know how many subscribers are there

And I don't think I can update the cron job with PHP, So that if all the subscribers received the emails stop executing the script.

User1804
  • 37
  • 7
  • Welcome to SO! Please include some code what you have already done, otherwise you risk that your question gets flag. For more, see https://stackoverflow.com/tour – B--rian Aug 08 '19 at 15:18
  • I don't think code is important here, I have a DB and I get the records then send emails to them – User1804 Aug 08 '19 at 15:23
  • I still suggest that you edit the question, so that it does contain enough details (Which DB? What did you exactly try? What server architecture? Which operating system? Why do you do that in PHP?). – B--rian Aug 08 '19 at 15:26
  • I added some code, The DB is MYSQL, It's a Cpanel, I do that with PHP because I have experience with PHP – User1804 Aug 08 '19 at 15:38
  • 1
    You really should farm bulk emailing out to a specialist service. DIY jobs risk getting black holed. – Quentin Aug 08 '19 at 15:41
  • @User1804: Thanks for the edit. Please put that information from the comment also inside the question. – B--rian Aug 08 '19 at 15:42
  • @Quentin, So there is no way to get it working by sending fewer emails each time using a cron job? – User1804 Aug 08 '19 at 17:13
  • @User1804 - I didn't say that. I just don't recommend DIY here. – Quentin Aug 08 '19 at 19:23
  • I updated the question with some ideas, Please check them and let me know if you have a better one – User1804 Aug 09 '19 at 11:05

1 Answers1

0

I recommend a few possible approaches.

One is to remove lines from the file as you process it

Another is to do it asynchronously, using a system like Kafka or RabbitMQ.

Finally, you can record the line number you are processing in persistent storage, read that, and jump to that line of the file.

Eric Hartford
  • 16,464
  • 4
  • 33
  • 50