49

I have a script that does a lot of legwork nightly.

It uses a PDO prepared statement that executes in a loop.

The first few are running fine, but then I get to a point where they all fail with the error: "MySQL server has gone away".

We run MySQL 5.0.77.

PHP Version 5.2.12

The rest of the site runs fine.

Nathan H
  • 48,033
  • 60
  • 165
  • 247
  • uhg we dont log mysql currently because we have way too much happening – Nathan H Feb 09 '10 at 20:05
  • Well its kind of hard to troubleshoot in that case as its a problem with mysql not necessarily PDO/PHP. I would turn it on run a sample batch of what runs overnight and then review the logs :-) – prodigitalson Feb 09 '10 at 20:10

9 Answers9

34

Most likely you sent a packet to the server that is longer than the maximum allowed packet.

When you try to insert a BLOB that exceeds your server's maximum packet size, even on a local server you will see the following error message on clientside:

MySQL server has gone away

And the following error message in the server log: (if error logging is enabled)

Error 1153 Got a packet bigger than 'max_allowed_packet' bytes

To fix this, you need to decide what is the size of the largest BLOB that you will ever insert, and set max_allowed_packet in my.ini accordingly, for example:

[mysqld]
...
max_allowed_packet = 200M
...
rustyx
  • 80,671
  • 25
  • 200
  • 267
32

The B.5.2.9. MySQL server has gone away section of the MySQL manual has a list of possible causes for this error.

Maybe you are in one of those situations ? -- Especially considering you are running a long operation, the point about wait_timeout might be interesting...

Pascal MARTIN
  • 395,085
  • 80
  • 655
  • 663
  • I think you are right. my wait timeout is 90 seconds and my code can sometime take a while before executing. Is there a way to change dynamically via PHP just for 1 script? – Nathan H Feb 09 '10 at 20:21
  • 4
    or a way to "ping" mysql via php PDO? – Nathan H Feb 09 '10 at 20:24
  • Judging from the doc, the simplest way seems to just re-connect ;; i.e. if you get that error, re-instanciating your PDO class might work – Pascal MARTIN Feb 09 '10 at 20:42
  • but wouldn't that kill my PreparedStatement (as it is attached to a PDO instance)? I am currently testing a work-around: sending "SELECT 1;" once in a while. – Nathan H Feb 09 '10 at 20:44
  • If you are sending queries like your `select 1` from another PHP script, it will probably not change a thing : each process has a separate connection to the database ;; and you quite can't do any kind of multi-threading in PHP ;; so I don't really seen another option than re-connecting when the connection is gone (and, yes, you'll have to re-prepare your statements :-( ) – Pascal MARTIN Feb 09 '10 at 20:48
  • No I am sending "select 1" from the same script, the same PDO instance. So far it's wokring. I wanted mysql_ping, but it seems PDO doesn't have it. So I guess "select 1" is the poor man's mysql_ping. – Nathan H Feb 09 '10 at 20:51
  • 11
    You can do $pdo->query("SET wait_timeout=1200;"); to set a specific timeout limit for your long-running script – Rafa Nov 09 '11 at 13:19
  • 2
    $pdo->exec("SET @@session.wait_timeout = {$waitTimeout}"); – Ross Apr 17 '13 at 14:21
  • This should be a comment as it does point to a documentation but has no solution to the problem. – Daniel W. May 11 '16 at 12:46
3

I had the same problem where the hosting server administration kills connection if there is a timeout.

Since I have used the query in major part I wrote a code which instead of using PDO class we can include the below class and replace the classname to "ConnectionManagerPDO". I just wrapped the PDO class.

final class ConnectionManagerPDO
{

    private $dsn;
    private $username;
    private $passwd;
    private $options;
    private $db;
    private $shouldReconnect;

    const RETRY_ATTEMPTS = 3;

    public function __construct($dsn, $username, $passwd, $options = array())
    {
        $this->dsn = $dsn;
        $this->username = $username;
        $this->passwd = $passwd;
        $this->options = $options;
        $this->shouldReconnect = true;
        try {
            $this->connect();
        } catch (PDOException $e) {
            throw $e;
        }
    }

    /**
     * @param $method
     * @param $args
     * @return mixed
     * @throws Exception
     * @throws PDOException
     */
    public function __call($method, $args)
    {
        $has_gone_away = false;
        $retry_attempt = 0;
        try_again:
        try {

            if (is_callable(array($this->db, $method))) {

                return call_user_func_array(array($this->db, $method), $args);
            } else {

                trigger_error("Call to undefined method '{$method}'");
                /*
                 * or
                 *
                 * throw new Exception("Call to undefined method.");
                 *
                 */
            }
        } catch (\PDOException $e) {

            $exception_message = $e->getMessage();

            if (
                ($this->shouldReconnect)
                && strpos($exception_message, 'server has gone away') !== false
                && $retry_attempt <= self::RETRY_ATTEMPTS
            ) {
                $has_gone_away = true;
            } else {
                /*
                 * What are you going to do with it... Throw it back.. FIRE IN THE HOLE
                 */
                throw $e;
            }
        }

        if ($has_gone_away) {
            $retry_attempt++;
            $this->reconnect();
            goto try_again;
        }
    }


    /**
     * Connects to DB
     */
    private function connect()
    {
        $this->db = new PDO($this->dsn, $this->username, $this->passwd, $this->options);
        /*
         * I am manually setting to catch error as exception so that the connection lost can be handled.
         */
        $this->db->setAttribute( PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION );
    }

    /**
     * Reconnects to DB
     */
    private function reconnect()
    {
        $this->db = null;
        $this->connect();
    }
}

Then use can start using the above class as you do in PDO.

try {
    $db = new ConnectionManagerPDO("mysql:host=localhost;dbname=dummy_test", "root", "");
    $query = $db->query("select * from test");
    $query->setFetchMode(PDO::FETCH_ASSOC);
}
catch(PDOException $e){
    /*
        handle the exception throw in ConnectionManagerPDO
    */
}
nhahtdh
  • 55,989
  • 15
  • 126
  • 162
mysticmo
  • 47
  • 3
  • The automatic reconnect didn't work with me. So I removed the __call function and made the reconnect function public. I call it when I do a query in a try/catch block. – Mel_T Oct 16 '15 at 12:30
  • It will only reconnect if pdo returns an exception "server has gone away". Can you please tell me what exception does it occur. – mysticmo Oct 20 '15 at 14:59
  • I tried again: The exception was "MySQL server has gone away" and it did make the reconnect, that's fine. But the query which should have been executed the moment the exception occured is lost - so I had to try/catch the query to be able to execute it again after the reconnect. – Mel_T Oct 22 '15 at 06:54
  • Interesting! Happy that you where able to achieve it. – mysticmo Oct 22 '15 at 08:02
  • 3
    Down - because using goto. Very bad practice – Ernestas Stankevičius Feb 01 '17 at 14:01
  • Can it cause a infinite interaction? – LeonanCarvalho Nov 24 '17 at 13:46
2

Try using PDO::setAttribute(PDO::ATTR_EMULATE_PREPARES, true) on your pod instance(s). Dont know that it will help but with no log data its all i got.

prodigitalson
  • 60,050
  • 10
  • 100
  • 114
2

It's likely that either your connection has been killed (e.g. by wait_timeout or another thread issuing a KILL command), the server has crashed or you've violated the mysql protocol in some way.

The latter is likely to be a bug in PDO, which is extremely likely if you're using server-side prepared statements or multi-results (hint: Don't)

A server crash will need to be investigated; look at the server logs.

If you still don't know what's going on, use a network packet dumper (e.g. tcpdump) to dump out the contents of the connection.

You can also enable the general query log - but do it very carefully in production.

MarkR
  • 62,604
  • 14
  • 116
  • 151
2

Nathan H, below is php class for pdo reconnection + code usage sample. Screenshot is attached.

<?php

# set errors reporting level
error_reporting(E_ALL ^ E_NOTICE ^ E_WARNING);

# set pdo connection
include('db.connection.pdo.php');

/* # this is "db.connection.pdo.php" content
define('DB_HOST', 'localhost');
define('DB_NAME', '');
define('DB_USER', '');
define('DB_PWD', '');
define('DB_PREFIX', '');
define('DB_SHOW_ERRORS', 1);

# connect to db
try {
    $dbh = new PDO('mysql:host='.DB_HOST.';dbname='.DB_NAME, DB_USER, DB_PWD);
    $dbh->setAttribute(PDO::ATTR_DEFAULT_FETCH_MODE, PDO::FETCH_ASSOC);
    $dbh->setAttribute(PDO::ATTR_EMULATE_PREPARES, TRUE);
    $dbh->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
}
catch(PDOException $e) {
    # echo $e->getMessage()."<br />";
    # exit;
    exit("Site is temporary unavailable."); #
}
*/

$reconnection = new PDOReconnection($dbh);

$reconnection->getTimeout();

echo $dbh->query('select 1')->fetchColumn();
echo PHP_EOL;
echo 'sleep 10 seconds..'.PHP_EOL;

sleep(10);

$dbh = $reconnection->checkConnection();
echo $dbh->query('select 1')->fetchColumn();
echo PHP_EOL;
echo 'sleep 35 seconds..'.PHP_EOL;

sleep(35);

$dbh = $reconnection->checkConnection();
echo $dbh->query('select 1')->fetchColumn();
echo PHP_EOL;
echo 'sleep 55 seconds..'.PHP_EOL;

sleep(55);

$dbh = $reconnection->checkConnection();
echo $dbh->query('select 1')->fetchColumn();
echo PHP_EOL;

echo 'sleep 300 seconds..'.PHP_EOL;
sleep(300);
$dbh = $reconnection->checkConnection();
echo $dbh->query('select 1')->fetchColumn();
echo PHP_EOL;

# *************************************************************************************************
# Class for PDO reconnection
class PDOReconnection
{
    private $dbh;

    # constructor
    public function __construct($dbh)
    {
        $this->dbh = $dbh;
    }

    # *************************************************************************************************

    # get mysql variable "wait_timeout" value
    public function getTimeout()
    {
        $timeout = $this->dbh->query('show variables like "wait_timeout"')->fetch(); # print_r($timeout);
        echo '========================'.PHP_EOL.'mysql variable "wait_timeout": '.$timeout['Value'].' seconds.'.PHP_EOL.'========================'.PHP_EOL;
    }

    # *************************************************************************************************

    # check mysql connection
    public function checkConnection()
    {
        try {
            $this->dbh->query('select 1')->fetchColumn();
            echo 'old connection works..'.PHP_EOL.'========================'.PHP_EOL;
        } catch (PDOException $Exception) {
            # echo 'there is no connection.'.PHP_EOL;
            $this->dbh = $this->reconnect();
            echo 'connection was lost, reconnect..'.PHP_EOL.'========================'.PHP_EOL;
        }

        return $this->dbh;
    }

    # *************************************************************************************************

    # reconnect to mysql
    public function reconnect()
    {
        $dbh = new PDO('mysql:host=' . DB_HOST . ';dbname=' . DB_NAME, DB_USER, DB_PWD);
        $dbh->setAttribute(PDO::ATTR_DEFAULT_FETCH_MODE, PDO::FETCH_ASSOC);
        $dbh->setAttribute(PDO::ATTR_EMULATE_PREPARES, TRUE); 
        $dbh->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);    
        return $dbh;
    }
}
# /Class for PDO reconnection
# *************************************************************************************************
0

I got this same error this morning after changing my DB properties in Laravel. I'd commented out the old settings and pasted in new ones. The problem was that the new settings where missing the DB_CONNECTION variable:

DB_CONNECTION=pgsql

Obviously you need to add whatever connection type you are using: sqlite, mysql, ...

maccaroo
  • 819
  • 2
  • 12
  • 22
-1

I had the exact same problem. I resolved this issue by doing unset on the PDO object instead of seting it to NULL.

For example:

function connectdb($dsn,$username,$password,$driver_options) {
    try {
        $dbh = new PDO($dsn,$username,$password,$driver_options);
        return $dbh;
    }
    catch(PDOException $e)
    {
        print "DB Error: ".$e->getMessage()."<br />";
        die();
    }
    
}

function closedb(&$dbh) {
    unset($dbh);             // use this line instead of $dbh = NULL;
}

Also, it is highly recommended to unset all your PDO objects. That includes variables that contain prepared statements.

gsmari
  • 17
  • 3
  • 8
    Aren't you just unsetting the local variable inside `closedb`? i.e., that function does nothing. `$dbh = null` would do nothing too unless you slap an `&` in the function signature. – mpen Aug 02 '16 at 19:12
  • thanks for pointing that out @mpen, I have now added the missing & to pass by reference – gsmari Oct 19 '21 at 10:57
  • Still won't work. https://ideone.com/ioJKku See also https://stackoverflow.com/a/10261948/65387 – mpen Oct 20 '21 at 01:26
-4
$pdo = new PDO(
    $dsn,
    $config['username'],
    $config['password'],
    array(
        PDO::ATTR_PERSISTENT => true,
        PDO::ATTR_ERRMODE    => PDO::ERRMODE_EXCEPTION
    )
);

try this. It may work

j0k
  • 22,600
  • 28
  • 79
  • 90