0

Suppose I'm a PHP application and I start an innodb transaction with mysqli_begin_transaction and it returns true. Then I do a series of inserts. As far as I am aware --- as a PHP application ---, all inserts went in perfectly. (Assume I checked all possible error codes.) It is my understanding that before I commit, the entire transaction could be rolled back --- due to a deadlock or something. Suppose that before I commit I delay a period of time doing something not MySQL related. Suppose further that --- during this delay --- MySQL gets into a situation where it must roll back my entire transaction. I claim I wouldn't know that it was rolled back because I'm not at this moment invoking any MySQL procedure. (Would the MySQL driver in PHP somehow discover so and throw an exception? I fear it would not, but I don't know.) When I'm ready to commit, it seems to me that MySQL will have no transaction pending relative to my connection, so I will be committing a new and empty transaction. MySQL reports no error when I commit an empty transaction, so I have no way of noticing that my entire transaction was rolled back.

Question. What is a small sample of code where I can be sure that the responsibility of my data in completely in MySQL's hands and not on mine? (In other words, how do I make sure (using PHP and MySQL) that when I commit a transaction, the entire transaction is successfully in MySQL's hands?)

Reference. Previously, I used to trust the return boolean from a mysqli_commit. It is my understanding now that I cannot trust it. The answer in this linked-question might be saying that all I need is that try-catch with mysqli_report not silenced. But I'm not convinced. Since I cannot easily reproduce an implicit rollback, I'm looking for two things --- (1) a sample of code where I can be sure the commit takes place as expected, (2) a deterministic way to reproduce an implicit rollback, if possible. It seems to me that (2) is not easy, so I'm researching (1) first.

Versions. I'm running MySQL 5.7.38-log and PHP 7.4.

Dharman
  • 30,962
  • 25
  • 85
  • 135
  • Why do you no longer trust the returned boolean value from mysqli_commit()? The notes are about error reporting which is different. I would be very surprised if you couldn't just assume all is well if it returns true. – Simon Goater Feb 23 '23 at 16:29
  • I would be very surprised as well. However, this is what happened. See my comments below about my theory --- I wonder if somehow we lost the transaction and when the `COMMIT` went in, it was a new and empty transaction, so MySQL simply said okay to the `COMMIT`. (That would explain our [logs](https://stackoverflow.com/questions/75534496).) – user21266319 Feb 26 '23 at 11:06
  • Have you seen a case where all the statements were successful and the `COMMIT` was successful _and_ there were no hardware failures, yet the changes to the database vanished? – Rick James Feb 26 '23 at 17:22
  • I can't say much about hardware failures --- these are MySQL databases stored in AWS. I would be very surprised if what we have (apparently more than once, given the reports from customer service) is due to hardware failures. But what all of our evidence points to here is that the statements were successful because I have the IDs of the inserts and the `COMMIT` was successful because PHP's procedure returned `TRUE`. (We logged.) (We did not check for the return of `START TRANSACTION`, though.) We have now new code that will make such checks. (What do you think?) – user21266319 Feb 27 '23 at 13:21
  • I think you are still avoiding enabling error reporting. If you can't add it to the whole project, then just add `mysqli_report(MYSQLI_REPORT_ERROR | MYSQLI_REPORT_STRICT);` before `begin_transaction()` and `mysqli_report(MYSQLI_REPORT_OFF);` after `commit()` – Dharman Feb 27 '23 at 13:34
  • Yes, I'm still avoiding. This setting seems to be applied at the MySQLi driver-level, so if I enable it --- even for a restricted period while the procedure under analysis runs ---, it will affect all code that's running in that period too. I'm afraid it could crash --- the programmers who wrote the system never ever used `mysqli_report()`. Would you have something to say that might change my outlook? I'm all ears! Thank you! – user21266319 Feb 28 '23 at 00:34

2 Answers2

0

A DB transaction, as the name implies, happens in the database. PHP has no way of knowing what MySQL does unless MySQL informs the client (PHP) about it. MySQL can only inform the client when it performs an action on the server. So in this regard, communication between PHP and MySQL resembles HTTP communication where the client makes a request and expects a response.

DB transactions ensure that the data is committed as a single atomic operation (part of ACID). This means that all DML operations within a transaction are treated as a single DML operation even when made up of multiple DML commands. Either all of them succeed or none of them succeeds. You can trust that the database will not commit part of the data.

So as we have established in the previous two paragraphs, MySQL will always tell you whether an operation succeeded or failed and it will never commit part of the transaction, the question arises: when and how can PHP tell MySQL to commit the transaction? The answer is pretty simple: send COMMIT SQL command whenever all DML operations you wanted to be part of the transaction were executed successfully.

Using mysqli (it's pretty much the same in PDO), it would look something like this:

$mysqli->begin_transaction();
$stmt = $mysqli->prepare("INSERT INTO foo(text) VALUES(?)");
$stmt->execute(['bar']);
// more DML operations
$mysqli->commit();

That's it! If any of the prepared statements fail, PHP will throw an exception based on the error received as a response from MySQL. Just make sure you have mysqli error reporting enabled! If commit() fails, then the whole transaction fails (C in ACID).

Without error reporting enabled, you could have one of the queries fail without noticing it. If you'd proceed with a commit, you would end up committing changes without the one that failed.

The reason why we have a ROLLBACK command is that MySQL does not implicitly rollback the queries that were part of the transaction and have successfully been executed. As long as the connection session remains open, MySQL will wait for either COMMIT or ROLLBACK to end the transaction. If you would like to discard the changes and proceed with some other operations or even retry the whole transaction again using the same connection, you need to rollback the transaction. A try-catch in PHP is helpful to achieve this. For more information see my other answer: How to start and end transaction in mysqli?


A word of caution: DB transactions are ACID, but certain DDL commands trigger implicit commit. If you are unaware of this and you execute DDL command as part of your transaction, you could end the transaction prematurely and cause data inconsistency.

Dharman
  • 30,962
  • 25
  • 85
  • 135
  • Thank you for all your insights. There's no DDL involved. (Should have stated that in the question, though I was not aware of the fact.) Thank you for the analogy with HTTP. I was feeling it had to work like that, like I was not sure and now I can lean on your experience. The reason I'm interested in this problem is for my own education. I don't want to ever lose data --- and this has happened in this issue I'm working on. I'm now fully aware of the turning `MYSQLI_REPORT_ALL` on. – user21266319 Feb 26 '23 at 11:00
  • However, notice the `COMMIT` command was definitely sent and it definitely got an okay response from MySQL. So I need to somehow explain the possibility that even after a `COMMIT`, a transaction can be completely rolled back. (I don't believe in that. This would mean MySQL has a data loss possibility.) My belief is that somehow we lost the transaction before the `COMMIT`. When we sent the `COMMIT`, there was no transaction anymore and MySQL is happy to say okay for an empty `COMMIT`. That explains what we saw in the [logs](https://stackoverflow.com/questions/75534496). – user21266319 Feb 26 '23 at 11:03
  • 1
    If MySQL told you that the commit operation succeeded then your transaction was saved. It cannot be rolled back after commit. – Dharman Feb 26 '23 at 11:08
  • I'm glad to hear that. My situation, though, is that I got an okay commit and my data has been lost --- so it must be something local that's fooling us and not MySQL. That's what I'm going after. Thank you! – user21266319 Feb 27 '23 at 13:21
  • For the record, my team found two more such cases in February. (We got similar logs as we displayed in the previous question.) We're not sure what to do, but I'm glad the problem is not so infrequent. Any ideas you might have will be appreciated. (@RickJames, let us know if you have any further ideas too. Thank you!) – user21266319 Feb 28 '23 at 22:25
-1

Do turn off "auto-reconnect". If the network hiccups in the middle of your transaction, the already-performed DMLs will be rolled back. But, after the auto-reconnect, you may be in autocommit mode and each DML is its own transaction.

Do test every DML and DDL statement. Do test after Commit, especially if using Galera and other clustering solutions.

In most cases, it is "correct" to jump back to the beginning of Begin and start over.

With MySQL 8.0, some DDL commands can be inside a transaction.

Rick James
  • 135,179
  • 13
  • 127
  • 222
  • Thank you! I was not aware of that. But `mysqli.reconnect` is currently off. (I've seen some of your answers and I have been reading your notes on your website. I'm glad to see you here. Please educate me on how to find a way to reproduce the problem I'm handling. I'm highly interested in it and I seem to be in the dark.) See the comments I added above to Dharman. They might illuminate you on figuring out a possibility for [what has happened here](https://stackoverflow.com/questions/75534496). We're not using Galera or any cluster solution. – user21266319 Feb 26 '23 at 11:04
  • People down-voted your answer here (go figure out!), but your mentioning `auto-reconnect` is definitely good thinking. So, thank you. – user21266319 Feb 28 '23 at 22:24