3

The following T-SQL query is taking 54 seconds to execute:

SELECT top (3)
   a.c1, b.c2, c.c3, d.c4 
FROM 
   table1 as a WITH (NOLOCK) 
JOIN 
   table2 as b WITH (NOLOCK) ON a.c1 = b.c4 
LEFT JOIN 
   table3 as c WITH (NOLOCK) ON a.c1 = c.c4 
LEFT JOIN 
   table4 as d WITH (NOLOCK) ON b.c3 = d.c1 
WHERE 
   b.source = '8R' 
ORDER BY 
   b.RecvdDate ASC

Whereas the same query sorted in descending order execute within a second

SELECT top (3)
   a.c1, b.c2, c.c3, d.c4 
FROM 
   table1 as a WITH (NOLOCK) 
JOIN 
   table2 as b WITH (NOLOCK) ON a.c1 = b.c4 
LEFT JOIN 
   table3 as c WITH (NOLOCK) ON a.c1 = c.c4 
LEFT JOIN 
   table4 as d WITH (NOLOCK) ON b.c3 = d.c1 
WHERE 
   b.source = '8R' 
ORDER BY 
   b.RecvdDate DESC

So how can I improve the performance of the query for both the sort orders?

I am using Microsoft SQL Server 2012 (SP1) - 11.0.3128.0 (X64) Standard Edition (64-bit) on Windows NT 6.2 (Build 9200: )

Execution Plan using SET STATISTICS PROFILE ON as follows:

Rows                 Executes             StmtText                                                                                                                                                                                                                                                         StmtId      NodeId      Parent      PhysicalOp                     LogicalOp                      Argument                                                                                                                                                                                                                                                         DefinedValues                                                                         EstimateRows  EstimateIO    EstimateCPU   AvgRowSize  TotalSubtreeCost OutputList                                                                                                                       Warnings                        Type                                                             Parallel EstimateExecutions
-------------------- -------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ----------- ----------- ----------- ------------------------------ ------------------------------ ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------- ------------- ------------- ------------- ----------- ---------------- -------------------------------------------------------------------------------------------------------------------------------- ------------------------------- ---------------------------------------------------------------- -------- ------------------
3                    1                    SELECT top 3 a.c1, b.c2, c.c3, d.c4  
FROM table1 as a WITH (NOLOCK) JOIN table2 as b WITH (NOLOCK) ON a.c1 = b.c1 LEFT JOIN table3 as c WITH (NOLOCK) ON 1           1           0           NULL                           NULL                           NULL                                                                                                                                                                                                                                                             NULL                                                                                  3             NULL          NULL          NULL        0.1093194        NULL                                                                                                                             NULL                            SELECT                                                           0        NULL
3                    1                      |--Top(TOP EXPRESSION:((3)))                                                                                                                                                                                                                                   1           2           1           Top                            Top                            TOP EXPRESSION:((3))                                                                                                                                                                                                                                             NULL                                                                                  3             0             3E-07         93          0.1093194        [a].[c1], [table2].[c2], [c].[c3], [d].[c4]                                                     NULL                            PLAN_ROW                                                         0        1
3                    1                           |--Nested Loops(Left Outer Join, WHERE:([DB].[dbo].[table3].[c3] as [table3].[c3]=[DB].[dbo].[table4].[c1] as [d].[c1]))                                                                                1           3           2           Nested Loops                   Left Outer Join                WHERE:([DB].[dbo].[table2].[c3] as [table2].[c3]=[DB].[dbo].[table4].[id] as [d].[id])                                                                                                                         NULL                                                                                  3             0             221.8893      101         0.1093191        [a].[c1], [table2].[c2], [table2].[RecvdDate], [c].[c3], [d].[c4]               NULL                            PLAN_ROW                                                         0        1
3                    1                                |--Nested Loops(Left Outer Join, OUTER REFERENCES:([a].[c1], [Expr1012]) WITH ORDERED PREFETCH)                                                                                                                                          1           4           3           Nested Loops                   Left Outer Join                OUTER REFERENCES:([a].[c1], [Expr1012]) WITH ORDERED PREFETCH                                                                                                                                                                                        NULL                                                                                  3             0             18.49077      99          0.1057252        [a].[c1], [table2].[c2], [table2].[RecvdDate], [table2].[C3], [c].[c3]  NULL                            PLAN_ROW                                                         0        1
3                    1                                |    |--Nested Loops(Inner Join, OUTER REFERENCES:([table2].[c4], [Expr1011]) WITH ORDERED PREFETCH)                                                                                                                                  1           6           4           Nested Loops                   Inner Join                     OUTER REFERENCES:([table2].[c4], [Expr1011]) WITH ORDERED PREFETCH                                                                                                                                                                                NULL                                                                                  3             0             18.78465      31          0.09258068       [a].[c1], [table2].[c2], [table2].[RecvdDate], [table2].[C3]               NULL                            PLAN_ROW                                                         0        1
27                   1                                |    |    |--Nested Loops(Inner Join, OUTER REFERENCES:([table2].[c1], [Expr1010]) WITH ORDERED PREFETCH)                                                                                                                            1           8           6           Nested Loops                   Inner Join                     OUTER REFERENCES:([table2].[c1], [Expr1010]) WITH ORDERED PREFETCH                                                                                                                                                                               NULL                                                                                  22.09678      0             136.1955      41          0.07926979       [table2].[c4], [table2].[c2], [table2].[RecvdDate], [table2].[C3]       NULL                            PLAN_ROW                                                         0        1
19928681             1                                |    |    |    |--Index Scan(OBJECT:([DB].[dbo].[table2].[IX_datereceived] AS [table2]), ORDERED BACKWARD)                                                                                                     1           10          8           Index Scan                     Index Scan                     OBJECT:([DB].[dbo].[table2].[IX_datereceived] AS [table2]), ORDERED BACKWARD                                                                                                                                               [table2].[c1], [table2].[RecvdDate]                   22.09678      66.78609      35.84109      19          0.003349548      [table2].[c1], [table2].[RecvdDate]                                                              NULL                            PLAN_ROW                                                         0        1
27                   19928681                         |    |    |    |--Clustered Index Seek(OBJECT:([DB].[dbo].[table2].[PK_c1] AS [table2]), SEEK:([table2].[c1]=[DB].[dbo].[table2].[c1] as [table2].[c1]),   1           12          8           Clustered Index Seek           Clustered Index Seek           OBJECT:([DB].[dbo].[table2].[PK_c1] AS [table2]), SEEK:([table2].[c1]=[DB].[dbo].[table2].[c1] as [table2].[c1]),  WHERE:([DB].[dbo].[table2].[Source]  [table2].[c4], [table2].[c2], [table2].[C3]  3.047679      0.003125      0.0001581     77          0.07582787       [table2].[c4], [table2].[c2], [table2].[C3]                                             NULL                            PLAN_ROW                                                         0        23.09678
3                    3                                |    |    |--Index Seek(OBJECT:([DB].[dbo].[table1].[PK_c1] AS [a]), SEEK:([a].[c1]=[DB].[dbo].[table2].[c4] as [table2].[c4]) ORDERED FORWARD)                                  1           20          6           Index Seek                     Index Seek                     OBJECT:([DB].[dbo].[table1].[PK_c1] AS [a]), SEEK:([a].[c1]=[DB].[dbo].[table2].[c4] as [table2].[c4]) ORDERED FORWARD                                                                       [a].[c1]                                                                  1             0.003125      0.0001581     11          0.02297952       [a].[c1]                                                                                                             NULL                            PLAN_ROW                                                         0        7.000085
3                    3                                |    |--Clustered Index Seek(OBJECT:([DB].[dbo].[table3].[table3_PK] AS [c]), SEEK:([c].[c4]=[DB].[dbo].[table1].[c1] as [a].[c1]) ORDERED FORWARD)                                    1           21          4           Clustered Index Seek           Clustered Index Seek           OBJECT:([DB].[dbo].[table3].[table3_PK] AS [c]), SEEK:([c].[c4]=[DB].[dbo].[table1].[c1] as [a].[c1]) ORDERED FORWARD                                                                              [c].[c3]                                                                           1             0.003125      0.0001581     75          0.01313197       [c].[c3]                                                                                                                      NULL                            PLAN_ROW                                                         0        4
36                   3                                |--Table Scan(OBJECT:([DB].[dbo].[table4] AS [d]))                                                                                                                                                                                    1           22          3           Table Scan                     Table Scan                     OBJECT:([DB].[dbo].[table4] AS [d])                                                                                                                                                                                                               [d].[id], [d].[c4]                                                            12            0.0032035     9.17E-05      17          0.0035703        [d].[id], [d].[c4]                                                                                                       NO STATS:([table4].[c1])  PLAN_ROW                                                         0        3.999999

(11 row(s) affected)

Link to execution plan in xml format:

Martin Smith
  • 438,706
  • 87
  • 741
  • 845
Leejoy
  • 1,356
  • 5
  • 23
  • 36
  • 3
    We'll need to see the query plans (actual QP is better than the estimated QP) Save them as *.SQLPLAN files into some internet accessible storage and then add links to them here. – RBarryYoung Feb 21 '14 at 16:01
  • And the table definitions, including all available indexes. Plus a description of the data behaviour (as would be shown in entity relationship diagram). And anything else you can give. – MatBailie Feb 21 '14 at 16:02
  • If you are only selecting the first 3 rows, could you add another `where` condition to restrict `RecvdDate` to, say, the last 5 days? That would discard most of the rows that you don't need. – David R Tribble Feb 21 '14 at 19:28

2 Answers2

2

table2 has 32,582,700 rows.

The estimated plan looks as follows.

enter image description here

You have a non covering index on RecvdDate and the plan has key lookups to retrieve the missing columns and evaluate the predicate on source = '8R'.

SQL Server estimates that it will need to do 22 -23 such lookups before it finds the TOP 3 matching rows and it can exit.

This assumes that rows matching the 8R predicate are scattered evenly with respect to date. In your case they are not and they are all on later dates. It it is actually doing 19,928,681 lookups in reality (from STATISTICS PROFILE output in question) rather than the 20 or so estimated.

The easiest way to fix it will be to supply an index on source, RecvdDate. As far as this query goes it doesn't matter what direction. Any of the four possibilities would work.

source asc, RecvdDate asc
source asc, RecvdDate desc
source desc, RecvdDate asc
source desc, RecvdDate desc

It can still do an equality seek on source and traverse the matching rows forward or backwards as required to get RecvdDate ASC or RecvdDate DESC.

Martin Smith
  • 438,706
  • 87
  • 741
  • 845
  • I have supplied an index as you suggested and it helped me to solve the issue. Thank you very much.. :) – Leejoy Feb 21 '14 at 19:38
0

Is that performance consistent? With that big a different my guess is that you have a composite index on

source, RecvdDate DESC

which means that it can't be used when querying by RecvdDate in ascending order.

You could add separate indices for source and RecvdDate which would improve performance (with a single column index the order does not matter), or add another composite index with the RecvdDate in ascending order.

D Stanley
  • 149,601
  • 11
  • 178
  • 240
  • Found this question that verifies this theory. http://stackoverflow.com/questions/743858/sql-server-indexes-ascending-or-descending-what-difference-does-it-make – TTeeple Feb 21 '14 at 16:05
  • @Leejoy do you have a composite index similar to what I'm assuming? – D Stanley Feb 21 '14 at 16:06
  • Bit of a guess this though! Could equally be that all the rows matching `source = '8R'` happen to have later dates. – Martin Smith Feb 21 '14 at 16:13
  • Actually I don't see this at all anyway. An equality seek on `source` can be scanned either forwards or backwards. – Martin Smith Feb 21 '14 at 16:16
  • @MartinSmith I could be wrong, but I don't think that would affect the performance - it would either be an index seek on `source` then a scan on `RecvdDate` or a scan on `RecvdDate` plus then a scan on `source` - noether of which would be affected by the _direction_ of a `RecvdDate` index. – D Stanley Feb 21 '14 at 16:16
  • @DStanley - It's quite a common issue where there is a non covering index and it chooses a plan with lookups that it underestimates the number of lookups required before it hits the `TOP N`. It assumes that the rows matching will be scattered evenly in the index. If they are all clumped together at one end the estimated number of lookups can be way off. – Martin Smith Feb 21 '14 at 16:17
  • The index info I got using EXEC sp_helpindex table2 is: IX_table2_recvdDate nonclustered located on INDEXES RecvdDate(-) IX_table2_source nonclustered located on PRIMARY Source – Leejoy Feb 21 '14 at 16:18
  • 1
    @Leejoy - Post the execution plans if you want anything other than guesses. – Martin Smith Feb 21 '14 at 16:21
  • @Leejoy - Can you upload the XML version? That is very difficult to read as the alignment is all messed up and it has less information than the XML anyway. – Martin Smith Feb 21 '14 at 17:15
  • @Leejoy - Looks pretty much as I thought. It is scanning the index on `RecvdDate` backwards and doing a key lookup to evaluate the residual predicate on `[Source]=N'8R'`. It estimates it will only need to do 23 such lookups before it finds the top 5 matching rows. The table has 32,582,700 rows and probably this estimate is catastrophically wrong. You haven't supplied the **actual** execution plan (only the estimated) so we can't see how many lookups happen in reality. – Martin Smith Feb 21 '14 at 19:03
  • The easiest way to fix it will be to supply an index on `source, RecvdDate`. Doesn't matter what direction. It can do an equality seek on `source` and traverse the matching rows in either direction. – Martin Smith Feb 21 '14 at 19:08