15

I have a Stored Procedure (SP from now on) that inserts data to the database (SaveClient, see below). When the SP is done I redirect the PHP page to a different PHP page that lists the entries (FetchObjectList, see below). The list does not return the newly created record until I then reload/refresh the page.

The stored procedure has a COMMIT at the end, I close the database connection in the PHP code after the SP is called and there is a check for errors but nothing goes wrong.

The page itself returns a 200 statuscode which means it isn't cached so can't be browserrelated either.

The current workaround is a sleep(1) in the PHP code but when the code goes live I have no idea if it will suffice. I'd ofcourse rather have MySQL dish out the correct resultset.

EDIT: I'm using the MySQLi object interface of PHP, might be useful to know. ;)

My devcomputer got PHP 5.2.17, MySQL 5.0.51a (InnoDB) and Apache 2.2.17 installed and running on Windows 7 x64.

UPDATE

Added the following line CALL FetchObjectList('client_tbl', NULL, NULL, 1, 'client_tbl.name ASC', NULL, NULL); to the end of SaveClient. The resultset does not have the newly created client in the presented resultset.

UPDATE 2

I tried using the SQL_NO_CACHE as seen here but to no avail.

I will now try the same SQL directly in PHP instead of calling the SPs.

UPDATE 3 - 20 september

I've tried any reasonable answer/comment I've got so far without any luck. I tried to update my PHP and MySQL version today (since I today learned that the live server will run on PHP 5.3.something and MySQL 5.1.something) but did not get it to work. I need to update the PHP to get a more recent php_mysqli.dll/libmysql.dll since the one I got has only supports up to 5.0.51a and there might be my problem since nothing in the actual DB has worked. I tried the libmysql.dll from the MySQL install to no avail.

Note that I also changed the PHP code that I've included since I actually copied the wrong one that was calling the user_tbl and not the client_tbl and also simplified it (removed multiqueries) but still the same result.

I don't know what will happen to the bounty, if it reverts back to me I'll add it again.

Stored Procedure SaveClient

DELIMITER //

DROP PROCEDURE IF EXISTS work.SaveClient//

CREATE PROCEDURE work.SaveClient(
        IN ObjectID INT,
        IN UserID INT,
        IN ClientName VARCHAR(60),
        IN VersionFrom DATETIME,
        IN VersionTo DATETIME)
root:BEGIN

DECLARE EXIT HANDLER FOR SQLEXCEPTION ROLLBACK;

/* 
    Default values ---------------------------------------------------------------------------------------------------------
*/
    # Used to block INSERT/UPDATEs
    SET @DoChanges      = TRUE;

    SET @Fields     = '*';
    SET @Version        = NULL;
    SET @UserVersion    = NULL;
    SET @DateNow        = NOW();
    SET @VersionActive  = CONCAT(
        '( ( NOW() BETWEEN ', 
        'version_from AND ', 
        'version_to ) OR ( ', 
        'version_from < NOW() AND ', 
        'version_to IS NULL ) )'
    );

    IF VersionFrom IS NULL THEN
        SET VersionFrom = @DateNow;
    END IF;

/*
    Search for client ------------------------------------------------------------------------------------------------------
*/
    IF ObjectID IS NOT NULL THEN
        SET @Client = CONCAT(
            'SELECT version INTO @Version FROM client_tbl WHERE object_id = ',
            ObjectID,
            ' AND ',
            @VersionActive
        );
        PREPARE stmt FROM @Client;
        EXECUTE stmt;
        DEALLOCATE PREPARE stmt;

        # Check if there are any changes
        IF @Version IS NOT NULL THEN
            SELECT name INTO @Name FROM client_tbl WHERE name = ClientName AND version = @Version;
            IF @Name = ClientName THEN 
                SET @errorMsg = "Duplicate entry";
                SET @errorCode = "S0000002";
                SELECT @errorCode, @errorMsg;
                LEAVE root;
            END IF;
        END IF;

    END IF;
/*
    Search for user ---------------------------------------------------------------------------------------------------------
*/
    # Create this as a function
    IF UserID IS NOT NULL THEN
        SET @User = CONCAT(
            'SELECT version INTO @UserVersion FROM user_tbl WHERE object_id = ',
            UserID,
            ' AND ',
            @VersionActive
        );
        PREPARE stmt FROM @User;
        EXECUTE stmt;
        DEALLOCATE PREPARE stmt;
    END IF;

    IF @UserVersion IS NULL THEN
        SET @errorMsg = "User is missing";
        SET @errorCode = "U0000099";
        SELECT @errorCode, @errorMsg;
        LEAVE root;
    END IF;

/*
    Add the client ---------------------------------------------------------------------------------------------------------
*/
    # Close the current version
    IF @Version IS NOT NULL THEN 
        IF @DoChanges = TRUE THEN 
            CALL UpdateVersion(
                ObjectID, 
                UserID, 
                @Version, 
                @DateNow, 
                'client_tbl'
            );
            SET @Version = @Version + 1;
        END IF;
    ELSE
        SET @Version = 1;
    END IF;

    IF @DoChanges = TRUE THEN 

        IF ObjectID IS NULL THEN
            INSERT INTO 
                object_tbl 
                (
                    object_class_id, 
                    created,
                    created_by
                )
                VALUES(
                    2,
                    NOW(),
                    UserID
                )
            ;
            SET ObjectID = LAST_INSERT_ID();
        END IF;

        INSERT INTO 
            client_tbl 
            (
                object_id, 
                version, 
                version_from, 
                version_to, 
                changed, 
                changed_by, 
                name
            ) 
            VALUES(
                ObjectID,
                @Version,
                VersionFrom,
                NULL,
                @DateNow,
                UserID,
                ClientName
            )
        ;
    END IF;

    COMMIT;
END //

DELIMITER ;

Stored Procedure FetchObjectList

DELIMITER //

DROP PROCEDURE IF EXISTS work.FetchObjectList//

CREATE PROCEDURE work.FetchObjectList(
        IN ObjectType VARCHAR(60),
        IN ObjectSubType VARCHAR(60),
        IN ObjectSubID INT,
        IN IsActive INT,
        IN OrderBy VARCHAR(100),
        IN SetStart INT,
        IN MaxResults INT)
root:BEGIN

DECLARE EXIT HANDLER FOR SQLEXCEPTION ROLLBACK;

    # Allow the "JSON" output be a max of 8kb
    SET GLOBAL group_concat_max_len = 8096;

/* 
    Default values ---------------------------------------------------------------------------------------------------------
*/
    SET @Fields     = '*';
    SET @VersionWhere   = '1'; # Get everything
    SET @Special        = '';
    SET @OrderBy        = '';
    SET @SetStart       = '';
    SET @MaxResults     = '';
    SET @JoinIn     = '';

    IF IsActive = 1 THEN
        SET @VersionWhere = CONCAT(
            '( NOW() BETWEEN ', 
            ObjectType, 
            '.version_from AND ', 
            ObjectType, 
            '.version_to OR ( ', 
            ObjectType, 
            '.version_from < NOW() AND ', 
            ObjectType, 
            '.version_to IS NULL ) )'
        );
    END IF;

    IF OrderBy != '' THEN
        SET @OrderBy = CONCAT(
                'ORDER BY ', 
                OrderBy
        );
    END IF;

/*
    Specials for each type -------------------------------------------------------------------------------------------------
*/

/*
    - Clients ------------
*/

    IF ObjectType = 'client_tbl' THEN
        SET @Fields = '
            *, 
            client_tbl.object_id AS object_id, 
            (
                SELECT 
                    COUNT(*) AS Total 
                FROM 
                    client_user_privilege_tbl cup 
                WHERE 
                    cup.client_id = client_tbl.object_id 

            ) AS usercount
        ';
    END IF;
/*
    - Configuration ------------
*/

    IF ObjectType = 'configuration_tbl' THEN
        SET @Fields = '
            *
        ';
    END IF;
/*
    Add upp the query to run -----------------------------------------------------------------------------------------------
*/
    SET @Query = CONCAT(
        'SELECT ',
        @Fields,
        ' FROM ', 
        ObjectType, 
        ' ',
        @JoinIn, 
        ' WHERE ', 
        @VersionWhere,
        ' ',
        @Special, 
        @OrderBy

    );

    PREPARE stmt FROM @Query;

    EXECUTE stmt;

    DEALLOCATE PREPARE stmt;

    COMMIT;

END //

DELIMITER ;

PHP CODE SNIPPET (Updated 20 september)

$query = "CALL FetchObjectList('client_tbl', NULL, NULL,  1, NULL, NULL, NULL)";
addTrace($query);

$rs = $db->query($query);
if( $rs ) {
    addTrace('Query done -> Results: ' . $rs->num_rows);
    while($r = $rs->fetch_assoc()){
        $fetchArray[] = $r;
    }
    $count = $rs->num_rows;
    $rs->close();
    $db->next_result();
} else {
    addTrace('Query failed -> ' . $db->error);
    flushTrace();
    exit;
}
Community
  • 1
  • 1
Henrik Ammer
  • 1,889
  • 13
  • 26
  • I have a Stored Procedure that saves data to the database. When I run another Stored Procedure on the page that loads after the insert (new page is loaded after you insert once? or page is just reloaded?), the result is empty. If I reload the page, the result is there. Please edit the first sentence, as you know better what actually happens – Sami Aug 26 '12 at 15:11
  • Added an update where I used the SP inside the SaveClient SP. Same result, the new record is not in the resultset but stored in the database nonetheless. – Henrik Ammer Aug 26 '12 at 15:52
  • Can you verify that the database changed after the SP? (With HeidiSQL or PhpMyAdmin etc.) Maybe it's just a cache problem on the second page. Try adding a random GET parameter. – Wolfgang Stengel Aug 26 '12 at 15:52
  • 1
    Yes. Now You deserved vote for the clarity. Now problem is understandable as well. Just do one thing and Your problem is caught. Do what? Execute stored procedure (You will go to redirected page automatically) Do not refresh the page check values in database are they there? Off-course they would be. – Sami Aug 26 '12 at 15:52
  • If you find values in above condition then you have to share the php code of redirected page only where you are fetching the value through stored procedure. Actually mistake is most likely there. – Sami Aug 26 '12 at 15:54
  • @WolfgangStengel Yes, the data is there. And as noted earlier, the resulting page sends a statuscode that states that is updated and not cached. Even so, the test with adding a timestampnumber in the URL creates the same error, latest entry not in the resultset returned. – Henrik Ammer Aug 26 '12 at 16:03
  • could you post some php code? can you test it on newer versions of mysql and php? – mrok Aug 26 '12 at 21:07
  • @mrok I'll update it with the PHP code used when I get home from work in about 9-10 hours or so. Regarding the update; I can update MySQL to 5.0.95 but not PHP since that is what the live environment is running. – Henrik Ammer Aug 27 '12 at 06:52
  • @mrok Finally updated with PHP code that uses the SP and returns it into an array. Had a ton of work to do so was first now I had the time. Any thoughts? – Henrik Ammer Aug 28 '12 at 22:16
  • Can this be related to caching? Read up on the following post, http://stackoverflow.com/questions/181894/mysql-force-not-to-use-cache-for-testing-speed-of-query. I'll try later today. – Henrik Ammer Aug 30 '12 at 07:18
  • @HenrikAmmer - I do not know the correct answer, but I would try combination of query instead of multi_query and closing/opening new connection for every call. It could produce unbuffered query error, but maybe during test something interesting occur. – mrok Sep 03 '12 at 23:20
  • @mrok But if the savefunction is called on a different page then the one showing the list, is that not enough to ensure that buffering is not occuring, or? I'll try the cache thing now though (as I found before). – Henrik Ammer Sep 04 '12 at 17:10
  • based on documentation http://dev.mysql.com/doc/refman/5.1/en/query-cache.html there should be no cache for prepared statement in you db version. – mrok Sep 04 '12 at 17:50
  • Yeah, that did not help, just tried it. I will hopefully soon get word of the upcoming upgrade of MySQL on the liveserver so I can upgrade aswell and see if the issue is solved. Also for tonight I'll check what happens when the queries are called directly in PHP without the SP if that changes anything. – Henrik Ammer Sep 04 '12 at 17:56
  • not an uber MySQLer but is it possible you have a write-behind configuration on the database, and the writes are still in the buffer waiting to write to disc? To the above notes, I think your suspect is the database handling of the read/writes. – Dave Sep 07 '12 at 19:13
  • @Dave How do I check this? I tried searching for some information but most was about how to enable it and nothing could be reverse-engineered back to what I have locally. – Henrik Ammer Sep 07 '12 at 19:23
  • A few wild ideas: (1) I don't see a START TRANSACTION in your SP. (2) try issuing a START TRANSACTION and COMMIT from PHP wrapped around your calls to your stored procedure (you never know..) (3) Try explicitly closing the DB connection from PHP before sending the redirect. EDIT: maybe not so helpful, I noticed you already are closing the connection... – Levi Sep 07 '12 at 22:53
  • @Levi 1) `BEGIN` is an alias of `START TRANSACTION`. 2) No difference. 3) Yes, it's closed. :'(. Hopefully I will get some good news on being able to upgrade today. Crossing fingers! – Henrik Ammer Sep 10 '12 at 16:24

2 Answers2

1

Since it's a pretty aged version of mysql it would not surprise me that this would be bug related but one thing I would -in your place- want to know is if this would work by not using transactions at all. (e.g. autocommit = on).

For that version 5.0 I would also check the query cache and disable it all together instead of per query (see SHOW VARIABLES LIKE 'have_query_cache'; SET GLOBAL query_cache_size =0; ). That would at the least eliminate those playing a role in this problem, reproduce(or try) the problem and see if anything changed. If not, I would start searching for specific bugs, especially when the query cache is disabled and it still does this without using transactions.

I verified the support for 5.0 mysql (innodb).

  • innodb_flush_log_at_trx_commit = 1
  • innodb_flush_method = O_DIRECT

Set these options specifically in your my.cnf, the top one is most important. They explain nicely what those do.

See http://dev.mysql.com/doc/refman/5.0/en/innodb-parameters.html#sysvar_innodb_flush_log_at_trx_commit

Glenn Plas
  • 1,608
  • 15
  • 17
  • I've disabled the query cache and removed all transaction SQL from the SP, checked that `autocommit` is ON and I also have `innodb_flush_log_at_trx_commit=1`. `O_DIRECT` is not available on Windows though. – Henrik Ammer Sep 20 '12 at 20:24
  • Indeed, my bad, that's not a windows option indeed. Amal Jacob has a point though concerning multiple result sets. – Glenn Plas Sep 21 '12 at 06:45
  • I don't have multiple resultsets from neither `SaveClient` nor `FetchObjectList`. `SaveClient` only returns a resultset if I found a duplicate entry and `FetchObjectList` only returns one. On the webpage itself I now have limited it down to only run one query/CALL but still have the issue. And it wasn't your bad, you are trying to help me and I appreciate it. – Henrik Ammer Sep 21 '12 at 09:26
  • At this point I would switch on the general log and look what is actually being done/returned. http://dev.mysql.com/doc/refman/5.0/en/query-log.html This problem intrigues me now. I think Amal Jacob is still right, I remember hitting this problem in the past, there is a hidden resultset of the SP itself like he mentions, so yours + an empty one = multi , you should be able to see that in the general log afaik. – Glenn Plas Sep 22 '12 at 23:43
  • I see this: 120923 17:42:07 30 Connect user@localhost on database 30 Query CALL FetchObjectList('configuration_tbl', NULL, NULL, NULL, NULL, NULL, NULL) 30 Query CALL SaveClient(NULL,1,'TuffTuff', NULL, NULL) What intrigues me is that it doesn't say 30 Quit like all other pages does. I think we are on to something here. – Henrik Ammer Sep 23 '12 at 15:44
  • I've tried to both close and kill the connection through PHP `$db->close()` and `$db->kill(threadid)` but still no `Quit` after the `SaveClient` call. – Henrik Ammer Sep 24 '12 at 07:18
  • Is it possible to edit your post to include the table schema's of the tables involved, I'd like to give this a try to reproduce, I'm sure it will bring back memories to the similar case I solved 7yrs ago ;-) – Glenn Plas Sep 24 '12 at 16:18
  • I checked some docs and for mysqli it seems you need to do something like : // MYSQLI: calling sp returning a recordset $rs = $mysqli->query( ‘CALL FetchObjectList('client_tbl', NULL, NULL, 1, NULL, NULL, NULL)’ ); while($row = $rs->fetch_object()) { print_r($row); } instead of fetch_assoc. see sources here: http://www.joeyrivera.com/2009/using-mysql-stored-procedures-with-php-mysqlmysqlipdo/ and here http://php.net/manual/en/mysqli.quickstart.stored-procedures.php they seem to use multi_query to call an SP from mysqli. – Glenn Plas Sep 24 '12 at 16:25
1

The problem with mysql_query is that it does not support multiple resultsets. And that is exactly what your stored procedures do – they tend to return more than one resultset. Whenever you call an SP, the exit status is secretly carried in an (empty) resultset along. If you add it up with your own output from a procedure, some of the resultsets from the query will be ignored by your PHP retrieval code. When you try to run another query, the pending resultset(s) will still be in the buffer.

Amal Jacob
  • 66
  • 3