Sendmail Exception reached when thousands emails sent in bulk mail script. Has sendmail a email-sent limit for a given process?

Question

I am a little desperate with a problem in our web server. Our web is a community with a lot of users and activity. We send to users personalized mails of the activity of their interest. We have a problem in the script that handles these mailings.

Our Bulk email script fails when it has thousands of emails to send. It usually works fine but when it has to send more than usual emails (aprox. 25.000 emails), it throws an Exception repeatedly:

Unable to send mail. mail():
Could not execute mail delivery program '/usr/sbin/sendmail -t -i '

The strange thing is that sendmail works properly in other processes such the web server, and sendmail is called in the same way in PHP (Zend). Sendmail only fails in the PHP bulk mail script when a lot emails have been already sent without errors. When the first exception is thrown, next calls to sendmail fail also. It seems as if some queue limit has been reached, but only for this process!

Code of PHP script

The PHP bulk mail scipt main loop is executed thousands of times. In each loop pass calls sendMail with a different $email and $user:

// Sometimes, hundred thousands iterations
foreach($notifications as $j => $notification){
    ...
    $mail->setNotification($notification);
    $this->sendMail($mail, $user);
    ...          
}

$this->sendmail($mail, $user) calls Zend internal method for sending mail. It calls PHP native method mail.

/**
 * Send mail using PHP native mail()
 *
 * @access public
 * @return void
 * @throws Zend_Mail_Transport_Exception if parameters is set
 *         but not a string
 * @throws Zend_Mail_Transport_Exception on mail() failure
 */
public function _sendMail()
{
    ...

        set_error_handler(array($this, '_handleMailErrors'));

        // CALL TO MAIL PHP NATIVE METHOD
        $result = mail(
            $this->recipients,
            $this->_mail->getSubject(),
            $this->body,
            $this->header,
            $this->parameters);
        restore_error_handler();
    }

    if ($this->_errstr !== null || !$result) {
        /**
         * @see Zend_Mail_Transport_Exception
         */
        require_once 'Zend/Mail/Transport/Exception.php';

        // HERE THE EXCEPTION IS THROWN
        throw new Zend_Mail_Transport_Exception('Unable to send mail. ' . $this->_errstr);
    }
}

Proccesses `sendmail` running

See the ps -aux | grep sendmail output when the bulk mail scipt is working fine

$ ps -aux | grep sendmail
root      6756  0.0  0.0  62240  2468 ?        Ss   18:19   0:08 sendmail: MTA: accepting connections          
root     25766  0.0  0.0  62668  3536 ?        Ss   22:43   0:00 sendmail: MTA: ./r17Lh1fX025764 eml4.in.gr.: client DATA status
root     30978  0.0  0.0  62460  2876 ?        Ss   22:46   0:00 sendmail: MTA: ./r17Lk8li030976 s1.m1r3.onet.pl.: user open
root     31462  0.0  0.0  62672  3536 ?        Ss   22:46   0:00 sendmail: MTA: ./r17LkSIg031460 mx2.hotmail.com.: client DATA status
root     31474  0.0  0.0  62672  3540 ?        Ss   22:46   0:00 sendmail: MTA: ./r17LkT54031472 mx2.hotmail.com.: client DATA status
root     31494  0.0  0.0  62668  4404 ?        Ss   22:46   0:00 sendmail: MTA: ./r17LkUXC031492 gmail-smtp-in.l.google.com.: client RCPT
root     31498  0.0  0.0  62668  3536 ?        Ss   22:46   0:00 sendmail: MTA: ./r17LkUn1031496 mx4.hotmail.com.: client DATA status
root     31502  0.0  0.0  62672  3536 ?        Ss   22:46   0:00 sendmail: MTA: ./r17LkUti031500 mx3.hotmail.com.: client DATA status
root     31506  0.0  0.0  62672  3500 ?        Ss   22:46   0:00 sendmail: MTA: ./r17LkUHw031504 mx4.hotmail.com.: client RCPT
root     31510  0.0  0.0  62672  3496 ?        Ss   22:46   0:00 sendmail: MTA: ./r17LkUth031508 mx3.hotmail.com.: client MAIL
root     31514  0.0  0.0  62668  4436 ?        Ss   22:46   0:00 sendmail: MTA: ./r17LkVPb031512 gmail-smtp-in.l.google.com.: client DATA status
root     31518  0.0  0.0  62460  2888 ?        Ss   22:46   0:00 sendmail: MTA: ./r17LkV9o031516 mx1.hotmail.com.: client EHLO
root     31522  0.0  0.0  62668  4404 ?        Ss   22:46   0:00 sendmail: MTA: ./r17LkVD4031520 gmail-smtp-in.l.google.com.: client RCPT
root     31526  0.0  0.0  62460  2852 ?        Ss   22:46   0:00 sendmail: MTA: ./r17LkVcF031524 mx3.hotmail.com.: user open

When the script starts throwing exceptions, ps -aux | grep sendmail outputs almost empty, as expected

$ ps -aux | grep sendmail
root      6756  0.0  0.0  62240  2468 ?        Ss   Feb07   0:49 sendmail: MTA: accepting connections

Some questions

I am quite newbie with sendmail so I appreciate any kind of help. If you need more info, please tell me.

Briefly, how does sendmail works for sending mail?
Does sendmail have a limit of mails to send (our server is dedicated and has not specific restrictions set by the hosting provider)?
Does sendmail have any queue for process that limits the number of mails that can be sent by a given process?
Million dollar question: why is the exception beeing thrown and how to solve it? :-)

Thanks in advance for your help!

Please do not cross-post to multiple Stack Exchange sites. Instead, select the single most appropriate site and post there. — Michael Hampton, Mar 05 '13 at 12:53

score 1 · Answer 1 · answered Mar 06 '13 at 08:17

1

You create one sendmail process per one email sent. 25_000 emails may create too many processes for your computer to handle.

P.S. You may try to ask your question at news:comp.mail.sendmail - add something to indicate that it is not "spammer in trouble" question.

answered Mar 06 '13 at 08:17

A. Filip

11
1

Thanks! I have rewritten the question. As you can see we are not spammers, or try not to be :-). Please, clarify me how can I ask at news:comp.mail.sendmail – Emilio Nicolás Mar 06 '13 at 12:04

score 1 · Answer 2 · answered Mar 06 '13 at 12:56

1

How does sendmail works for sending mail?

It is not a short story. Anyway to reduce number of sendmail processes you may configure your php to send via smtp protocol to port 25 on 127.0.0.1 (loopback interface). It will create only "owned by root" sendmail processes.

You should also define confDELAY_LA, confQUEUE_LA (delay

confDELAY_LA slows speed of accepting new mails when system load average is higher than configured limit. It may/will make your script run longer.

confQUEUE_LA skips "at once" delivery attempts - you most likely should increase number of queue runners processes.

In short: you push your sendmail server beyond limits standard simple configuration is assumed to handle with ease.

answered Mar 06 '13 at 12:56

AnFi

10,493
3
23
47

In our sendmail.mc: `FEATURE(\`conncontrol', \`nodelay', \`terminate')dnl` `FEATURE(\`ratecontrol', \`nodelay', \`terminate')dnl` – Emilio Nicolás Mar 06 '13 at 16:55
what changes would you suggest for our scenario? Thanks! – Emilio Nicolás Mar 06 '13 at 16:59
conncontrol & ratecontrol - it is not important if you have entries in access table (/etc/mail/access) turning it off for loopback addresses. e.g. "clientrate:127 0" and "clientcon:127 0". – AnFi Mar 06 '13 at 17:35
I would suggest 1) swithing php to smtp 127.0.0.1:25 2) Increasing (after some test) confQUEUE_LA in sendail configuration - I think many modern servers can handle higher loads especially Linux which computes LA in another way giving higher values 3) IF "mailq" would report many messages in queue THEN increase number of queue directories - http://www.sendmail.org/~gshapiro/8.10.Training/mqueue.html - use /var/spool/mqueue* with extra directories named /var/spool/mqueue1 /var/spool/mqueue2 .... ; Do not create more than 3 extra directories in first try – AnFi Mar 06 '13 at 17:53

Sendmail Exception reached when thousands emails sent in bulk mail script. Has sendmail a email-sent limit for a given process?

Code of PHP script

Proccesses sendmail running

Some questions

2 Answers2

Proccesses `sendmail` running