5

I'm working on a new service that will need to provide reporting data for regions. Each region will have its own db table and each db table will have some columns with the same name like ReportDate. The different tables will have a lot of different/unique columns which is why I created a separate table for each region. There will be about 10 region tables.

I'm considering using dynamic sql for my sproc so I can write a simple query and then run it generically against each table. This would be much quicker and easier to write and maintain then writing 10 separate queries that perform the same SQL against all tables.

Is SS pretty well optimized to handle this type of implementation? Do you see any potential performance issues? Are there any specific steps I should take to optimize performance for this implmentation? Does anyone here think dynamic sql is "evil" for one reason or another? I'm referring specifically to dynamic sql completely done within SS as opposed to an external app passing in dynamic sql statements.

user6806234
  • 57
  • 1
  • 4

4 Answers4

5

Dynamic Sql is becoming Angel Not Evil !!!.

Ten years ago, there were only two rules in Relational database: "always use stored procedures" and "never build dynamic SQL (Ad Hoc Query)".

But, the game has changed with the the revolution of ORM and the new generations of sql server starting from SQL server 2005 up to SQL server 2016 release.

It's not amazing to hear an advice like:

Don’t Fear Dynamic SQL. When done correctly, it can be used effectively

Dynamic SQL certainly has some downsides. But when used properly, it has many positive attributes, including the ability to enhance performance.

Conditions for dynamic SQL high performance :

  • Run dynamic sql using sp_executesql not EXEC.
  • Use parameters for where conditions (DON'T concatenate values)

sp_executesql will use cached plan to get more performance.

ADO.NET send Queries in the client as sp_executeSQL. You can view the generated SQL in SQL Profiler.

What ORM does is dynamically generating CRUD as dynamic SQL and map data into objects to speed development with good performance.

SQL Server has sp_executesql , which is a system stored-procedure that takes a SQL string with parameter markers and a variable number of arguments for the parameters.

So the client issues an RPC request to sp_executesql whenever it has to execute a parameterized statement.

The parameterized sp_executesql call is more likely to have a reusable cached plan.

The stored procedure sp_executesql become the standard procedure for executing queries as dynamic sql generated in ADO.NET even (update/insert/delete) starting from SQL Server 2005.

The contents of dynamic sql with parameter substitution are compiled and executed as an execution plan separate from the execution plan of the batch that called sp_executesql.

Excerpt from MSDN sp_executesql

sp_executesql can be used instead of stored procedures to execute a Transact-SQL statement many times when the change in parameter values to the statement is the only variation. Because the Transact-SQL statement itself remains constant and only the parameter values change, the SQL Server query optimizer is likely to reuse the execution plan it generates for the first execution.

The dynamic sql is executed as: EXECUTE sp_executesql @SQLString, @ParmDefinition, @SalesID = @IntVariable;

Who has the power of Generating dynamic sql and run with good performance?

In the client if we run the query:

  var ID=1;
  string commandText = "select * from products WHERE productid = @ID;";

With ADO.NET:

The generated query is:

 exec sp_executesql N'select * from products WHERE productid = @ID;',N'@ID int',@ID=11

Dapper (high performance ORM)

Dapper is a micro-ORM for .NET developed and used by the Stack Overflow team, focusing on raw performance as the primary aim.

The generated query is:

  exec sp_executesql N'select * from products WHERE productid = @ID;',N'@ID int',@ID=12

EntityFramework EF 6.3

the query:

var id=1;
Product p = db.Products.FirstOrDefault(t => t.ProductID == id);

The generate dynamic sql :

exec sp_executesql N'SELECT TOP (1) 
    [Extent1].[ProductID] AS [ProductID], 
    [Extent1].[ProductName] AS [ProductName], 
    [Extent1].[SupplierID] AS [SupplierID], 
    [Extent1].[CategoryID] AS [CategoryID], 
    [Extent1].[QuantityPerUnit] AS [QuantityPerUnit], 
    [Extent1].[UnitPrice] AS [UnitPrice], 
    [Extent1].[UnitsInStock] AS [UnitsInStock], 
    [Extent1].[UnitsOnOrder] AS [UnitsOnOrder], 
    [Extent1].[ReorderLevel] AS [ReorderLevel], 
    [Extent1].[Discontinued] AS [Discontinued]
    FROM [dbo].[Products] AS [Extent1]
    WHERE [Extent1].[ProductID] = @p__linq__0',N'@p__linq__0 int',@p__linq__0=1

and other tools use dynamic sql with high performance.

Conclusion: Dynamic sql can be used with good performance if it's parameterized and executed with sp_executesql

M.Hassan
  • 10,282
  • 5
  • 65
  • 84
3

This is a bit long for a comment.

Dynamic SQL is a very reasonable solution. In fact, it can prevent some performance problems that arise when SQL statements are optimized the first time a procedure is run. This can result in sub-optimal execution plans for other parameters.

My recommendation is to generate exactly the SQL you want. The additional overhead is for compiling the statement. Assuming you are doing complex queries, running the query should take much longer than compiling it.

As for dynamic SQL being evil, I suppose that is a matter of opinion. I have used it very effectively on multiple projects over many years.

Gordon Linoff
  • 1,242,037
  • 58
  • 646
  • 786
0

SQL Server has no issues with dynamic sql, the problems lie squarely with developers. Dynamic sql can be very useful and powerful, but there are a lot of nuances that need to be understood in order to have long-term success. Usually someone asking the kinds of basic questions that you are asking is not a person with a depth of background to navigate the pitfalls.

One thing that really stands out to me is the premise that your regions are so vastly different that you need to have them in completely different databases, yet you want to use the same stored procedure in all of them. That does not make any sense to me and I fear you are headed for a mess.

btberry
  • 377
  • 1
  • 7
  • The different tables for each rregion is scary sounding. Think of this query. Select SUM(sales) from Regions (all regions all sales). If all the regions are not in one table then it is hard to do basic reporting and gui utils. – Sql Surfer Sep 14 '16 at 03:23
0

The problem of storing similar-but-not-the-same objects in a relational DB is a very common one. Search for normalize+inheritance, or see here. I would avoid recreating the same columns in different tables. You have started to discover the problems. You're also heading down the road of putting conditional logic in SQL. I wouldn't be going there.

Dynamic SQL is not a problem. SQL in string literals is. Put your sql in stored procs if you want, but for me, SQL requests belong to the application, and should be packaged and versioned with the application. QueryFirst will help you preserve the integrity of your SQL, while compiling it into your app.

Community
  • 1
  • 1
bbsimonbb
  • 27,056
  • 15
  • 80
  • 110