SQL Server: stored procedure become very slow, raw SQL query is still very fast

Question

We are struggling with a strange problem: a stored procedure become extremely slow when raw SQL is executed fairly fast.

We have

• SQL Server 2008 R2 Express Edition SP1 10.50.2500.0 with several databases on it.
• a database (it's size is around 747Mb)
• a stored procedure which takes different parameters and does select among multiple tables from the database.

Code:

ALTER Procedure [dbo].[spGetMovieShortDataList](
   @MediaID int = null,
   @Rfa nvarchar(8) = null,
   @LicenseWindow nvarchar(8) = null,
   @OwnerID uniqueidentifier = null,
   @LicenseType nvarchar(max) = null,
   @PriceGroupID uniqueidentifier = null,
   @Format nvarchar(max) = null,
   @GenreID uniqueidentifier = null,
   @Title nvarchar(max) = null,
   @Actor nvarchar(max) = null,
   @ProductionCountryID uniqueidentifier = null,
   @DontReturnMoviesWithNoLicense bit = 0,
   @DontReturnNotReadyMovies bit = 0,
   @take int = 10,
   @skip int = 0,
   @order nvarchar(max) = null,
   @asc bit = 1)
as 
begin
  declare @SQLString nvarchar(max);
  declare @ascending nvarchar(5);

  declare @ParmDefinition nvarchar(max);
  set @ParmDefinition = '@MediaID int,

  declare @now DateTime;
  declare @Rfa nvarchar(8),
          @LicenseWindow nvarchar(8),
          @OwnerID uniqueidentifier,
          @LicenseType nvarchar(max),
          @PriceGroupID uniqueidentifier,
          @Format nvarchar(max),
          @GenreID uniqueidentifier,
          @Title nvarchar(max),
          @Actor nvarchar(max),
          @ProductionCountryID uniqueidentifier,
          @DontReturnMoviesWithNoLicense bit = 0,
          @DontReturnNotReadyMovies bit = 0,
          @take int,
          @skip int,
          @now DateTime';

   set @ascending = case when @asc = 1 then 'ASC' else 'DESC' end  
   set @now = GetDate();
   set @SQLString = 'SELECT distinct m.ID, m.EpisodNo, m.MediaID, p.Dubbed, pf.Format, t.OriginalTitle into #temp
                FROM Media m
                inner join Asset a1 on m.ID=a1.ID
                inner join Asset a2 on a1.ParentID=a2.ID
                inner join Asset a3 on a2.ParentID=a3.ID
                inner join Title t on t.ID = a3.ID
                inner join Product p on a2.ID = p.ID
                left join AssetReady ar on ar.AssetID = a1.ID
                left join License l on l.ProductID=p.ID
                left join ProductFormat pf on pf.ID = p.Format ' 
                + CASE WHEN @PriceGroupID IS NOT NULL THEN 
                    'left join LicenseToPriceGroup lpg on lpg.LicenseID = l.ID ' ELSE '' END
                + CASE WHEN @Title IS NOT NULL THEN 
                    'left join LanguageAsset la on la.AssetID = m.ID ' ELSE '' END
                + CASE WHEN @LicenseType IS NOT NULL THEN 
                    'left join LicenseType lt on lt.ID=l.LicenseTypeID ' ELSE '' END
                + CASE WHEN @Actor IS NOT NULL THEN 
                    'left join Cast c on c.AssetID = a1.ID ' ELSE '' END
                + CASE WHEN @GenreID IS NOT NULL THEN 
                    'left join ListToCountryToAsset lca on lca.AssetID=a1.ID ' ELSE '' END
                + CASE WHEN @ProductionCountryID IS NOT NULL THEN 
                    'left join ProductionCountryToAsset pca on pca.AssetID=t.ID ' ELSE '' END
                +
                'where (
                1 = case  
                    when @Rfa = ''All'' then 1
                    when @Rfa = ''Ready'' then ar.Rfa
                    when @Rfa = ''NotReady'' and (l.TbaWindowStart is null OR l.TbaWindowStart = 0) and ar.Rfa = 0 and ar.SkipRfa = 0 then 1
                    when @Rfa = ''Skipped'' and ar.SkipRfa = 1 then 1
                end) '
                + 
                CASE WHEN @LicenseWindow IS NOT NULL THEN
                'AND 
                1 = (case 
                    when (@LicenseWindow = 1 And (l.WindowEnd < @now and l.TbaWindowEnd = 0)) then 1
                    when (@LicenseWindow = 2 And (l.TbaWindowStart = 0 and l.WindowStart < @now and (l.TbaWindowEnd = 1 or l.WindowEnd > @now))) then 1
                    when (@LicenseWindow = 4 And ((l.TbaWindowStart = 1 or l.WindowStart > @now) and (l.TbaWindowEnd = 1 or l.WindowEnd > @now))) then 1
                    when (@LicenseWindow = 3 And ((l.WindowEnd < @now and l.TbaWindowEnd = 0) or (l.TbaWindowStart = 0 and l.WindowStart < @now and (l.TbaWindowEnd = 1 or l.WindowEnd > @now)))) then 1
                    when (@LicenseWindow = 5 And ((l.WindowEnd < @now and l.TbaWindowEnd = 0) or ((l.TbaWindowStart = 1 or l.WindowStart > @now) and (l.TbaWindowEnd = 1 or l.WindowEnd > @now)))) then 1
                    when (@LicenseWindow = 6 And ((l.TbaWindowStart = 0 and l.WindowStart < @now and (l.TbaWindowEnd = 1 or l.WindowEnd > @now)) or ((l.TbaWindowStart = 1 or l.WindowStart > @now) and (l.TbaWindowEnd = 1 or l.WindowEnd > @now)))) then 1
                    when ((@LicenseWindow = 7 Or @LicenseWindow = 0) And ((l.WindowEnd < @now and l.TbaWindowEnd = 0) or (l.TbaWindowStart = 0 and l.WindowStart < @now and (l.TbaWindowEnd = 1 or l.WindowEnd > @now)) or ((l.TbaWindowStart = 1 or l.WindowStart > @now) and (l.TbaWindowEnd = 1 or l.WindowEnd > @now)))) then 1 
                end) ' ELSE '' END
                + CASE WHEN @OwnerID IS NOT NULL THEN 
                    'AND (l.OwnerID = @OwnerID) ' ELSE '' END
                + CASE WHEN @MediaID IS NOT NULL THEN 
                    'AND (m.MediaID = @MediaID) ' ELSE '' END
                + CASE WHEN @LicenseType IS NOT NULL THEN 
                    'AND (lt.Name = @LicenseType) ' ELSE '' END
                + CASE WHEN @PriceGroupID IS NOT NULL THEN 
                    'AND (lpg.PriceGroupID = @PriceGroupID) ' ELSE '' END
                + CASE WHEN @Format IS NOT NULL THEN 
                    'AND (pf.Format = @Format) ' ELSE '' END
                + CASE WHEN @GenreID IS NOT NULL THEN 
                    'AND (lca.ListID = @GenreID) ' ELSE '' END
                + CASE WHEN @DontReturnMoviesWithNoLicense = 1 THEN 
                    'AND (l.ID is not null) ' ELSE '' END
                + CASE WHEN @Title IS NOT NULL THEN 
                    'AND (t.OriginalTitle like N''%' + @Title + '%'' OR la.LocalTitle like N''%' + @Title + '%'') ' ELSE '' END
                + CASE WHEN @Actor IS NOT NULL THEN 
                    'AND (rtrim(ltrim(replace(c.FirstName + '' '' + c.MiddleName + '' '' + c.LastName, ''  '', '' ''))) like ''%'' + rtrim(ltrim(replace(@Actor,''  '','' ''))) + ''%'') ' ELSE '' END
                + CASE WHEN @DontReturnNotReadyMovies = 1 THEN 
                    'AND ((ar.ID is not null) AND (ar.Ready = 1) AND (ar.CountryID = l.CountryID))' ELSE '' END
                + CASE WHEN @ProductionCountryID IS NOT NULL THEN 
                    'AND (pca.ProductionCountryID = @ProductionCountryID)' ELSE '' END
                    +               
                ' 
                select #temp.* ,ROW_NUMBER() over (order by ';
                if @order = 'Title' 
                begin
                    set @SQLString = @SQLString + 'OriginalTitle';
                end
                else if @order = 'MediaID' 
                begin
                    set @SQLString = @SQLString + 'MediaID';
                end
                else
                begin
                    set @SQLString = @SQLString + 'ID';
                end

                set @SQLString = @SQLString + ' ' + @ascending + '
                ) rn
                into #numbered
                from #temp

                declare @count int;
                select @count = MAX(#numbered.rn) from #numbered

                while (@skip >= @count )
                begin
                    set @skip = @skip - @take;
                end

                select ID, MediaID, EpisodNo, Dubbed, Format, OriginalTitle, @count TotalCount from #numbered
                where rn between @skip and @skip + @take

                drop table #temp    
                drop table #numbered';

                execute sp_executesql @SQLString,@ParmDefinition, @MediaID, @Rfa, @LicenseWindow, @OwnerID, @LicenseType, @PriceGroupID, @Format, @GenreID, 
                    @Title, @Actor, @ProductionCountryID, @DontReturnMoviesWithNoLicense,@DontReturnNotReadyMovies, @take, @skip, @now
            end

The stored procedure was working pretty good and fast (it's execution usually took 1-2 seconds).

Example of call

DBCC FREEPROCCACHE

EXEC    value = [dbo].[spGetMovieShortDataList]
        @LicenseWindow =N'1',
        @Rfa = N'NotReady',     
        @DontReturnMoviesWithNoLicense = False,
        @DontReturnNotReadyMovies = True,
        @take = 20,
        @skip = 0,
        @asc = False,
        @order = N'ID'

Basically during execution of the stored procedure the executed 3 SQL queries, the first Select Into query takes 99% of time.

This query is

declare @now DateTime;
set @now = GetDate();

SELECT DISTINCT 
   m.ID, m.EpisodNo, m.MediaID, p.Dubbed, pf.Format, t.OriginalTitle
FROM Media m
INNER JOIN Asset a1 ON m.ID = a1.ID
INNER JOIN Asset a2 ON a1.ParentID = a2.ID
INNER JOIN Asset a3 ON a2.ParentID = a3.ID
INNER JOIN Title t ON t.ID = a3.ID
INNER JOIN Product p ON a2.ID = p.ID
LEFT JOIN AssetReady ar ON ar.AssetID = a1.ID
LEFT JOIN License l on l.ProductID = p.ID
LEFT JOIN ProductFormat pf on pf.ID = p.Format 
WHERE
   ((l.TbaWindowStart is null OR l.TbaWindowStart = 0) 
    and ar.Rfa = 0 and ar.SkipRfa = 0)
   And (l.WindowEnd < @now and l.TbaWindowEnd = 0 )
   AND ((ar.ID is not null) AND (ar.Ready = 1) AND (ar.CountryID = l.CountryID)) 

This stored procedure, after massive data update on the database (a lot tables and rows were affected by the update, however DB size was almost unchanged, now it is 752 ) become to work extremely slow. Now it takes from 20 to 90 seconds.

If I take raw SQL query from the stored procedure - it is executed within 1-2 seconds.

We've tried:

1. the stored procedure is created with parameters

   SET ANSI_NULLS ON SET QUOTED_IDENTIFIER ON

2. recreate the stored procedure with parameter with recompile

3. execute the stored procedure after purging prod cache DBCC FREEPROCCACHE
4. move part of where clauses into the join part
5. reindex tables
6. update statistics for the tables from the query using statements like UPDATE STATISTICS Media WITH FULLSCAN

However the execution of the stored procedure is still >> 30 seconds.

But if I run the SQL query which is generated by the SP - it is executed for less than 2 seconds.

I've compared execution plans for SP and for the raw SQL - they are quite different. During execution of RAW SQL - the optimizer is using Merge Joins, but when we execute SP - it uses Hash Match (Inner Join), like there are no indexes.

• Execution Plan for RAW SQl - Fast
• Execution Plan for SP - Slow

If someone knows what could it be - please help. Thanks in advance!

Answer 1

Try using using the hint OPTIMIZE FOR UNKNOWN. If it works, this may be better than forcing a recompile every time. The problem is that, the most efficient query plan depends on the actual value of the date paramter being supplied. When compiling the SP, sql server has to make a guess on what actual values will be supplied, and it is likely making the wrong guess here. OPTIMIZE FOR UNKNOWN is meant for this exact problem.

At the end of your query, add

OPTION (OPTIMIZE FOR (@now UNKNOWN))

http://blogs.msdn.com/b/sqlprogrammability/archive/2008/11/26/optimize-for-unknown-a-little-known-sql-server-2008-feature.aspx

Answer 2

Since you are using sp_executesql recompiling the procedure, or clearing the cached plan for the procedure won't actually help, the query plan for the query executed via sp_executesql is cached separately to the stored procedure.

You either need to add the query hint WITH (RECOMPILE) to the sql executed, or clear the cache for that specific sql before executing it:

DECLARE @PlanHandle VARBINARY(64);

SELECT  @PlanHandle = cp.Plan_Handle
FROM    sys.dm_exec_cached_plans cp
        CROSS APPLY sys.dm_exec_sql_text(plan_handle) AS st
WHERE   st.text LIKE '%' + @SQLString;

DBCC FREEPROCCACHE (@PlanHandle); -- CLEAR THE CACHE FOR THIS QUERY

EXECUTE sp_executesql @SQLString,@ParmDefinition, @MediaID, @Rfa, @LicenseWindow, @OwnerID, @LicenseType, @PriceGroupID, @Format, @GenreID, 
    @Title, @Actor, @ProductionCountryID, @DontReturnMoviesWithNoLicense,@DontReturnNotReadyMovies, @take, @skip, @now;

This is of course irrelevant if when you executed DBCC FREEPROCCACHE you didn't pass any parameters and cleared the whole cache.

Answer 3

OPTIMIZE FOR (@parameter1 UNKNOWN, @parameter2 UNKNOWN,...)

worked wonders for me. I had exactly the same problem. I did not use any of the other hints or even: WITH (RECOMPILE)

Answer 4

You should consider these options:

1. Set index in all tables used
2. Use local parameters/variable
3. Add:” Option (recompile) “ at the end of each query where local parameters are used