10

Let's say I have to run the SQL query:

SELECT data FROM table WHERE condition1 AND condition2 AND condition3 AND condition4 

Is that any different than

SELECT data FROM table WHERE condition3 AND condition1 AND condition4 AND condition2

?


If it's not different:

I know from my own experience that condition1 is less expensive than condition2 is less expensive than condition3 is less expensive than condition4.

If any of the prior conditions are not met, the remaining conditions should never be checked. It wouldn't be immediately obvious to the optimizer, as stored functions are involved. How should I write a query that does this?

Martin Smith
  • 438,706
  • 87
  • 741
  • 845
ieatpizza
  • 464
  • 4
  • 14
  • 1
    "Is that any different than" --- No. – zerkms Jun 28 '15 at 10:32
  • I've noticed that MySQL 8 seems to have suddenly changed how short circuit evaluation works, and that the right hand side of the expression is now also (first) being evaluated in a query that was find in MySQL 7. – Matthew Lock Feb 09 '22 at 23:17

2 Answers2

5

In practice, MySQL probably evaluates the conditions in order -- assuming that you have only one table in the query. If the conditions are between tables, then all bets are off.

Your question is unclear in some respects. If some of the conditions can be resolved at compile-time, then they often will (see here). Once an expression evaluates to FALSE in a chain of ANDs, then it doesn't need to further evaluate expressions.

Without explicit documentation, you can fall back on the ANSI definition of evaluation order. Here is a question about that specific topic. This basically says that there is no guarantee on the order of evaluations. It does suggest that:

where ((((condition1 and condition2) and condition3) and condition4)

would guarantee evaluation in a particular order. I suspect that redundant parentheses would be dropped during the compile phase, and the ANSI condition is not quite as clear as it seems.

The only expression that guarantees order of evaluation in SQL is the case expression. Although I am not a fan of using case in the where clause you could do:

where (case when not condition1 then 0
            when not condition2 then 0
            when not condition3 then 0
            else not condition4
       end)

This would guarantee order of evaluation.

By the way, for the operators and functions in MySQL, the time for an operation is not going to matter, compared to the time for retrieving the rows. Of course, there are exceptions. In particular, any pattern matching on long strings is going to be rather expensive. And, calls to functions may be very expensive as well.

Community
  • 1
  • 1
Gordon Linoff
  • 1,242,037
  • 58
  • 646
  • 786
0

The place to start investigating is with explain, which will show you what the optimizer does for your particular query.

Several examples are available for study; these seem useful:

Given those tools, others state that MySQL does short-circuit queries:

  • MySQL 12.3.3 Logical Operators developer's comment says

    The documentation doesn't say anything about this, but it appears (based on some tests I just ran) that MySQL short-circuits evaluation of logical operators just like most other languages do.

although the related order-by may not be:

However, as noted in the MySQL documentation comments, the documentation itself is not explicit. You would have to use the appropriate tools and rely upon repeatable behavior of the optimizer to determine the actual state of affairs.

Community
  • 1
  • 1
Thomas Dickey
  • 51,086
  • 7
  • 70
  • 105
  • Doesn't answer the question at the moment. Does MySQL short circuit or not? – Martin Smith Jun 28 '15 at 10:31
  • 1
    It shows what the optimizer *does*. I'll expand slightly. – Thomas Dickey Jun 28 '15 at 10:32
  • The information is either in developer's comments or in documentation; I'm adding what I find. – Thomas Dickey Jun 28 '15 at 10:38
  • And what you found is wrong. Your answer is harmful (since it states what is simply not correct). – zerkms Jun 28 '15 at 10:38
  • It's hard to prove something that does not exist. You cannot prove lack of something. There is nothing about it in documentation since it's not implemented. – zerkms Jun 28 '15 at 10:42
  • @zerkms A single counter example would prove that it does not reliably evaluate predicates in order and short circuit when the first one is false. – Martin Smith Jun 28 '15 at 10:43
  • Either you *know* or you do *not*. – Thomas Dickey Jun 28 '15 at 10:43
  • I do know it. It's not there, and the links provided here are wrong (most of them confuse predicate simplification and short circuiting, which are not the same thing) – zerkms Jun 28 '15 at 10:44
  • @MartinSmith "A single counter example would prove that it does not reliably evaluate predicates in order and short circuit when the first one is false" --- prove that it exists. Whoever makes a positive statement must prove it. – zerkms Jun 28 '15 at 10:47
  • @zerkms So is your understanding that MySQL doesn't reliably short circuit? Or just that it is not documented to do so and thus can't be relied on? – Martin Smith Jun 28 '15 at 10:50
  • @MartinSmith I know that there is no such a thing as a short circuiting. There is predicate optimisation which you and guys on the provided links confuse it with http://dev.mysql.com/doc/refman/5.6/en/where-optimizations.html But those are **COMPLETELY DIFFERENT** optimisation techniques. You cannot use the terms interchangeably. "So is your understanding that MySQL doesn't reliably short circuit?" -- short circuiting makes **NO SENSE** for a declarative language, since it's imperative by the nature. – zerkms Jun 28 '15 at 10:51
  • 1
    I'm not using the terms interchangeably. To me short circuiting would not be continuing to evaluate conditions C and D if A or B were false. And yes I know that from a SQL perspective it doesn't make sense as SQL is declarative and "all at once" but we are talking about a specific implementation here. MySQL does lots of things that aren't truly relational. – Martin Smith Jun 28 '15 at 10:53
  • Well, that does not happen in mysql. The constant falsy expressions are eliminated during optimisation step that is not a short circuiting. And that elimination does not depend on the operands order (so it is not a "short circuiting" which operates by the order by definition). – zerkms Jun 28 '15 at 10:54
  • And what about the predicates that remain as they can't be determined at optimisation time? Are they short circuited or will MySQL doggedly evaluate all of them at runtime? – Martin Smith Jun 28 '15 at 10:59
  • "as they can't be determined at optimisation time?" --- if they cannot be eliminated - they will be applied to every row/index. "doggedly evaluate all of them at runtime" --- if it relies on runtime (either data or other environment) - how can you evaluate it statically? If you are particularly asking about short circuiting in the final `WHERE` clause - you cannot state anything about that: the final clause may look entirely different to what you have queried. So you cannot say that "your query is short circiuted" since it was rewritten by an optimizer. – zerkms Jun 28 '15 at 11:01
  • 1
    Who said anything about evaluating statically? The point of short circuiting is that you don't need to always evaluate them at all. If for row 1 it is found at runtime that A is false then no need to even evaluate B,C,D for that row. – Martin Smith Jun 28 '15 at 11:04
  • @MartinSmith the predicates applied to the data not necessary look the same as you have put them in a query. You cannot say your expression is short circuited after it was rewritten for optimisation purposes. So the order of original operands does not matter, since mysql is free to evaluate them in any order it wants. – zerkms Jun 28 '15 at 11:05
  • So is your argument then that MySQL can short circuit but just that it does so on the predicates after the optimisation stage? – Martin Smith Jun 28 '15 at 11:11
  • @MartinSmith without doubts the "final" expression might be short circuited (as well as any evaluations in CPU can be as well), since it's just a boolean expression. The point is that the order of operands has no influence on that process, hence you cannot say that the order of them "short circuits" anything. And returning to the original question the only valid answer is "No", whereas this answer puts some hope (and misleading links, which in some places make stupid assumptions based on lack of knowledge). – zerkms Jun 28 '15 at 11:12