3

I am maintaining a simple DB tool that executes CreateDatabase on our EF model, and uses SMO to run some .sql scripts.

It currently uses:

var svrConnection = new ServerConnection(sqlConnection);
var server = new Server(svrConnection);
server.ConnectionContext.ExecuteNonQuery(fullSqlScript);

Is there a way to execute a TSQL script in .Net without using SMO?

Or, is there a way to successfully use SMO in an application without installing the thing on the server I'm running it on?

Any alternative that will be useful to me will require no installation on the box, besides xcopy of assemblies for my tool. It also must guarantee that the script will operate in exactly the same way without additional testing/verification.

The scripts use GO, etc, and cannot/should not be broken up - they're generated by third-party tools (aspnet_regsql.exe), and by hand (but are stale at this point), so I do not want to touch them if I can avoid it.

I'm pretty sure the suggestions in the comments on this answer are wrong, because I'm pretty sure GO isn't directly replacable with ;. If I'm wrong, please let me know :)

I'm trying to work around this problem:

Could not load file or assembly 'Microsoft.SqlServer.SqlClrProvider, Version=10.0.0.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91' or one of its dependencies. The system cannot find the file specified.

The fix mentioned here was to install SMO, which requires other packages, which isn't ideal for me.


Final solution, based off Russell McClure's answer:

In the end I am going to end up with sqlcmd.exe, since scripts written for it (anything with GO in it) could do way too much to replicate through SqlCommand.ExecuteNonQuery in my own code. And since it has the same level of dependencies as SMO, and would take some work to wrap in a programatically clean way, I'm just going to stick with SMO.

All the libraries and tools I mentioned are part of :

http://www.microsoft.com/download/en/details.aspx?id=16978

Community
  • 1
  • 1
Merlyn Morgan-Graham
  • 58,163
  • 16
  • 128
  • 183

4 Answers4

2

Run your T-SQL through sqlcmd.exe.

Or, what I normally do is actually read the file contents (my sql files are normally embedded resources) and then pass it to this function to split it into batches acceptable to ADO.NET:

public static string[] ParseSqlStatementBatch(string sqlStatementBatch)
{
   // split the sql into seperate batches by dividing on the GO statement
   Regex sqlStatementBatchSplitter = new Regex(@"^\s*GO\s*\r?$", RegexOptions.Multiline | RegexOptions.IgnoreCase);
   return sqlStatementBatchSplitter.Split(sqlStatementBatch);
}

Here is an example usage:

string[] sqlStatements = DataAccess.DatabaseWrapper.ParseSqlStatementBatch(FileContents);

using (SqlConnection sqlConnection = new SqlConnection(connectionString))
{
   sqlConnection.Open();
   using (SqlCommand command = sqlConnection.CreateCommand())
   {
      command.CommandType = CommandType.Text;
      command.CommandTimeout = Registry.RegistryWrapper.GetSqlCommandTimeout();
      foreach (string sqlStatement in sqlStatements)
      {
         if (sqlStatement.Length > 0)
         {
            command.CommandText = sqlStatement;
            command.ExecuteNonQuery();
Russell McClure
  • 4,821
  • 1
  • 22
  • 22
  • Good suggestion, though what do I have to install to get it on the box? Can I xcopy something over? I just did a `where sqlcmd` on the server and came up empty. – Merlyn Morgan-Graham Nov 09 '11 at 23:40
  • I guess I should have asked which version of SQL Server you were using. I think it comes with 2005 or later. I'm not sure which installation options end up putting it down. It's always been there for me. – Russell McClure Nov 09 '11 at 23:50
  • That's the trick - I am running this on a clean box (a build/deployment server) to deploy the DB remotely. I can play in my xcopy sandbox, but I'd rather avoid installing things on the server because that's more steps to replicate the box whenever we flatten it. – Merlyn Morgan-Graham Nov 09 '11 at 23:55
  • Though I've found the answer for my question in the comments here. I'm using SQL Server 2008 R2, and I found [what I have to install here](http://www.microsoft.com/download/en/details.aspx?id=16978#SNAC) and [here](http://www.microsoft.com/download/en/details.aspx?id=16978#SQLCMD). I'll still upvote because this solution could work for someone else, and I'm surprised I didn't think of it before :) – Merlyn Morgan-Graham Nov 09 '11 at 23:57
  • Just read the code (assumed it was for `sqlcmd`). I'll give it a go and let you know how it went. Thanks! – Merlyn Morgan-Graham Nov 10 '11 at 00:00
  • I messed with my codebase a couple times, and after all refactors I found my code looked darn near identical to what is in this code :) I like the use of `Regex`, though this regex looks more correct to me: `@"^\s*GO\s*;?\s*(--.*)?$"`, what do you think? – Merlyn Morgan-Graham Nov 10 '11 at 01:23
  • Correction: `^\s*GO\s*;?\s*(?:--.*)?$` - had to make the final grouping not capture, otherwise split ignored it. – Merlyn Morgan-Graham Nov 10 '11 at 01:33
  • You are right, my regex could use some improvements. We have a T-SQL style that we follow and it works fine for that. I'll probably incorporate your more robust regex back into my code. Thanks. – Russell McClure Nov 10 '11 at 05:58
  • Only problem I found with it is that it left extra newlines/carriage returns appended to the next line. I tried appending the `\r?` to the end, but somehow it didn't remove it. Leaving it out didn't break anything tough, so I just left it out. – Merlyn Morgan-Graham Nov 10 '11 at 08:08
  • Turns out I have to use SqlCmd.exe. I don't trust any other option, because I don't want to have to make a class that attempts to reimplement that program. There's just way too much that it does for me to replicate it cleanly: http://www.yaldex.com/sql_server_tutorial_3/ch06lev1sec3.html - Your answer still wins tho ;) – Merlyn Morgan-Graham Nov 10 '11 at 22:18
  • Note that [`GO`](http://msdn.microsoft.com/en-us/library/ms188037.aspx) is more complicated, e.g. to execute the same batch twice, one can use `GO 2` in `SQLCMD`. – Christian.K Sep 05 '12 at 04:23
2

You can use ADO.NET. This way you don't have to depend on sqlcmd.exe.

Here is a very helpful class to handle your scripts (with GO, :setvar, $(MyVar), etc).

http://bitmugger.blogspot.com/2008_04_01_archive.html

I have been using a slightly modified version in production for some months now without problems. Here is my version (can't remember what I changed - maybe added some supported commands).

using System;
using System.Collections.Generic;
using System.Text;
using System.Data.SqlClient;

namespace GR.WIX
{
    /// <summary>
    /// Supports running a SQLCmd Mode style statement such as the output from a VS 2008 Database Team Edition Database Project
    /// Only a limited subset of the SQLCmd mode syntax is supported. Other gotchas.
    ///
    ///
    ///
    /// Supported Commands:
    ///     GO (note GO [N] will only be executed once not N times)
    ///     :setvar
    ///     $(MyVar)
    ///     :on error exit
    ///     :on error resume (only for SQL Errors)
    ///     :on error ignore (only for SQL Errors)
    ///
    /// The Following SQLCMD Commands are recognized but ignored. They will not crash your script if encountered but will be skipped.
    ///     :ED
    ///     :Error
    ///     :!!
    ///     :Perftrace
    ///     :Quit
    ///     :Exit
    ///     :Help
    ///     :XML
    ///     :r
    ///     :ServerList
    ///     :Listvar
    ///
    /// The following SQLCMD pre-defined variables are pre-defined by this class just like they are by SQLCMD
    /// The only difference is SQLCMD actually used and/or updated these variable. This class simply has them predefined
    /// with much the same values as SQLCMD did. The class allows you to change ALL variables (unlike SQLCMD) where some are
    /// read only.
    ///     SQLCMDUSER ""
    ///     SQLCMDPASSWORD
    ///     SQLCMDSERVER {Server Name}
    ///     SQLCMDWORKSTATION {Computer Name}
    ///     SQLCMDLOGINTIMEOUT {Connection Timeout}
    ///     SQLCMDDBNAME {Database Name}
    ///     SQLCMDHEADERS "0"
    ///     SQLCMDCOLSEP " "
    ///     SQLCMDCOLWIDTH "0"
    ///     SQLCMDPACKETSIZE "4096"
    ///     SQLCMDERRORLEVEL "0"
    ///     SQLCMDMAXVARTYPEWIDTH "256"
    ///     SQLCMDMAXFIXEDTYPEWIDTH "0"
    ///     SQLCMDEDITOR "edit.com"
    ///     SQLCMDINI ""
    ///
    /// The following pre-defnined variables ARE used by the class and their values when set are not ignored
    ///     SQLCMDSTATTIMEOUT "0"
    ///   
    /// One Additional Variable is defined so that scripts could potentially detect they are running in this class instead
    /// of SQLCmd.
    ///     SQLCMDREAL "0"
    /// </summary>
    public class ExecuteSqlCmdMode
    {
        #region Fields
        private readonly Dictionary<string, string> variables;
        private readonly List<string> lockedVariables;
        private ErrorMode errorMode;
        private readonly SqlConnection connection;
        private readonly List<string> ignoredCommands;
        private bool allowVariableOverwrites;
        #endregion Fields

        #region Properties
        /// <summary>
        /// Gets or sets a value indicating whether to allow variable overwrites.
        /// If True then even though a variable is specified externally it may be overwritten by :SetVar in the script. If False then the reverse
        /// variables specified externally superscede :setvar.
        /// Default = false
        /// </summary>
        /// <value>true if allow variable overwrites; otherwise, false.</value>
        public bool AllowVariableOverwrites
        {
            get { return allowVariableOverwrites; }
            set { allowVariableOverwrites = value; }
        }
        #endregion Properties

        #region Constructor
        /// <summary>
        /// Initializes a new instance of the <see cref="ExecuteSqlCmdMode"/> class.
        /// </summary>
        /// <param name="sqlConnection">The SQL conn.</param>
        public ExecuteSqlCmdMode(SqlConnection sqlConnection)
        {
            // Check for legal values
            if (sqlConnection == null)
            {
                throw new Exception("connection cannot be null");
            }

            // Set connection variable from supplied SQLConnection.
            connection = sqlConnection;

            // Load up the script variables.
            variables = new Dictionary<string, string>();
            variables.Add("SQLCMDUSER", "");
            variables.Add("SQLCMDPASSWORD", "");
            variables.Add("SQLCMDSERVER", sqlConnection.DataSource);
            variables.Add("SQLCMDWORKSTATION", sqlConnection.WorkstationId);
            variables.Add("SQLCMDDBNAME", sqlConnection.Database);
            variables.Add("SQLCMDLOGINTIMEOUT", sqlConnection.ConnectionTimeout.ToString());
            variables.Add("SQLCMDSTATTIMEOUT", "0");
            variables.Add("SQLCMDHEADERS", "0");
            variables.Add("SQLCMDCOLSEP", "");
            variables.Add("SQLCMDCOLWIDTH", "0");
            variables.Add("SQLCMDPACKETSIZE", "4096");
            variables.Add("SQLCMDERRORLEVEL", "0");
            variables.Add("SQLCMDMAXVARTYPEWIDTH", "256");
            variables.Add("SQLCMDMAXFIXEDTYPEWIDTH", "0");
            variables.Add("SQLCMDEDITOR", "edit.com");
            variables.Add("SQLCMDINI", "");
            variables.Add("SQLCMDREAL", "0");

            // Setup pre-locked variables.
            lockedVariables = new List<string>();
            lockedVariables.Add("SQLCMDREAL");

            // Setup the list of commands to be ignored.
            ignoredCommands = new List<string>();
            ignoredCommands.Add(":ED");
            ignoredCommands.Add(":ERROR");
            ignoredCommands.Add(":!!");
            ignoredCommands.Add(":PERFTRACE");
            ignoredCommands.Add(":QUIT");
            ignoredCommands.Add(":EXIT");
            ignoredCommands.Add(":HELP");
            ignoredCommands.Add(":XML");
            //ignoredCommands.Add(":R");
            ignoredCommands.Add(":SERVERLIST");
            ignoredCommands.Add(":LISTVAR");

            // Some other misc values.
            errorMode = ErrorMode.ErrExit;
            allowVariableOverwrites = false;
        }

        #endregion Constructor

        /// <summary>
        /// Sets a variable in advance of script execution.
        /// </summary>
        /// <param name="variableName">Name of the variable.</param>
        /// <param name="variableValue">The variable value.</param>
        public void SetVariable(string variableName, string variableValue)
        {
            variableName = variableName.Trim().ToUpper();
            if (variableName.Length == 0  || variableName.Contains(" "))
            {
                throw new Exception(string.Format("Variable name {0} cannot be blank or contain spaces", variableName));
            }

            // See if we already have this variable
            if (variables.ContainsKey(variableName))
            {
                variables[variableName] = variableValue;
            }
            else
            {
                variables.Add(variableName, variableValue);

                if (!allowVariableOverwrites)
                {
                    lockedVariables.Add(variableName);
                }
            }
        }

        /// <summary>
        /// Executes the specified SQL script.
        /// </summary>
        /// <param name="scriptToExecute">The SQL script to execute.</param>
        public List<Exception> Execute(string scriptToExecute)
        {
            var exceptions = new List<Exception>();
            var queryBlock = new StringBuilder();

            connection.Open();


            var scriptLines = (scriptToExecute.Replace(Environment.NewLine, "\n") + "\nGO\n").Split('\n');

            // Loop each line in the script
            for (var i = 0; i < scriptLines.GetUpperBound(0); i++)
            {
                // Prepare a specially modified version of the line for checking for commands.
                var ucaseLine = scriptLines[i].Replace("\t", " ").Trim().ToUpper() + " ";

                // See if it's one of the commands to be ignored.
                if (ignoredCommands.Contains(ucaseLine.Split(' ')[0]))
                {
                    // Just ignore this line.
                }
                else if (ucaseLine.StartsWith("GO "))
                {
                    // We have a GO line (everything after GO on the line is ignored). Execute the block
                    // we have gathered so far.
                    ExecuteBlock(queryBlock, exceptions);
                    // After a GO command, we reset our query.
                    queryBlock = new StringBuilder();
                }
                else if (ucaseLine.StartsWith(":SETVAR "))
                {
                    // We have found a SetVar line. Add (or update) the variable and its value to our list.
                    SetVariableValue(scriptLines[i]);
                }
                else if (ucaseLine.StartsWith(":ON ERROR "))
                {
                    // Handle :on error.
                    HandleOnErrorCommand(i, ucaseLine);
                }
                else if (ucaseLine.StartsWith(":R "))
                {
                    // TODO: Handle this case.
                }
                else
                {
                    // Regular SQL Line to have variables replaced on then added to SQLCmd for execution.

                    // Replace variables with its value for the line (if any).
                    var noVariableVersion = ReplaceVariablesWithValue(scriptLines[i]);

                    // Add it to the current block of code to execute.
                    queryBlock.AppendLine(noVariableVersion);
                }
            }
            return exceptions;
        }

        private string ReplaceVariablesWithValue(string temp)
        {
            if (temp.Length > 4 && temp.Contains("$("))
            {
                // Loop each variable to check the line for.
                foreach (var keyPair in variables)
                {
                    var searchFor = string.Format("$({0})", keyPair.Key);
                    var begPos = temp.ToUpper().IndexOf(searchFor);
                    while (begPos >= 0)
                    {
                        // Make the variable substitution
                        var endPos = begPos + searchFor.Length;
                        temp = temp.Substring(0, begPos) + keyPair.Value + temp.Substring(endPos, temp.Length - endPos);

                        // Calculate a new begPos
                        begPos = temp.ToUpper().IndexOf(string.Format(searchFor));
                    }
                }
            }
            return temp;
        }

        private void ExecuteBlock(StringBuilder sqlCommand, List<Exception> exceptions)
        {
            try
            {
                if (sqlCommand.Length > 0)
                {
                    // Attempt the SQL command.
                    using (var sqlComm = new SqlCommand(sqlCommand.ToString(), connection))
                    {
                        sqlComm.CommandTimeout = 120;
                        sqlComm.ExecuteNonQuery();
                    }
                }
            }
            catch (Exception ex)
            {
                if (errorMode != ErrorMode.ErrIgnore)
                {
                    throw new Exception("Error executing " + sqlCommand, ex);
                }

                exceptions.Add(new Exception("Error executing " + sqlCommand, ex));
            }
        }
        private void HandleOnErrorCommand(int i, string ucaseLine)
        {
            var temp = ucaseLine.Substring(10, ucaseLine.Length - 10).Trim();
            if (temp == "EXIT")
            {
                errorMode = ErrorMode.ErrExit;
            }
            else if (temp == "RESUME" || temp == "IGNORE")
            {
                errorMode = ErrorMode.ErrIgnore;
            }
            else
            {
                throw new Exception(string.Format("Unknown On Error mode '{0}' on line {1}", temp, i));
            }
        }

        private void SetVariableValue(string scriptLine)
        {
            var temp = scriptLine.Trim().Substring(8, scriptLine.Trim().Length - 8);
            var begPos = temp.IndexOf(" ");

            var varName = temp.Substring(0, begPos).Trim().ToUpper();
            var varValue = temp.Substring(begPos + 1, temp.Length - begPos - 1).Trim();
            if (varValue.StartsWith("\"") && varValue.EndsWith("\""))
            {
                varValue = varValue.Substring(1, varValue.Length - 2);
            }
            else
            {
                throw new Exception(string.Format("Improperly formatted :SetVar on the following line {0}.", scriptLine));
            }

            if (variables.ContainsKey(varName))
            {
                if (!lockedVariables.Contains(varName))
                {
                    variables[varName] = varValue;
                }
            }
            else
            {
                variables.Add(varName, varValue);
            }
        }
    }

    /// <summary>
    /// Legal values for the error mode
    /// Error mode controls what happens when a SQL Error occurs
    /// </summary>
    public enum ErrorMode
    {
        ErrExit,
        ErrIgnore
    }
}

Use it like this:

/// <summary>
        /// Executes the SQL script.
        /// </summary>
        /// <param name="serverName">Name of the server.</param>
        /// <param name="scriptPath">The path of the script to execute.</param>
        /// <param name="variables">The variables.</param>
        /// <returns></returns>
        private static void ExecuteSqlScript(string serverName, string scriptPath, Dictionary<string, string> variables)
        {
            using (var connection = new SqlConnection(string.Format(SqlConnectionFormat, serverName)))
            {
                var mode = new ExecuteSqlCmdMode(connection);

                // Add variables.
                foreach (var variable in variables)
                {
                    mode.SetVariable(variable.Key, variable.Value);
                }

                mode.Execute(FileToString(scriptPath));
            }
        }
joerage
  • 4,863
  • 4
  • 38
  • 48
  • +1; Thanks for the answer! Though this is exactly the type of thing I was worried about, where I couldn't be guaranteed that SQL scripts didn't contain a bunch of non-sql garbage that would have to be filtered out. This is the type of code I wouldn't want to use unless I knew there was an automated test suite run by someone who wrote test cases around an officially published TSQL grammar =P So, nothing wrong with your code, but it seems like it is blowing away my confidence in all answers here :) – Merlyn Morgan-Graham Nov 10 '11 at 20:08
1

My solution for this problem:

Could not load file or assembly 'Microsoft.SqlServer.SqlClrProvider, Version=10.0.0.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91' or one of its dependencies. The system cannot find the file specified

With release of FW 4.0 MS introduced new type dynamic. I've created a class which loads SMO dll in runtime using this type. Sample:

public dynamic Load(string assemblyName, string assemblyNamespace, bool instantiate, params     object[] constructorParams)
{
    dynamic dynamicAssembly = null;
    try
    {
       string assemblyPath = GetAssemblyPath(assemblyName);
       Assembly assembly = Assembly.LoadFrom(assemblyPath);
       if (instantiate)
       {
           Type assemblyType = assembly.GetType(assemblyNamespace);
           if (constructorParams == null)
              dynamicAssembly = Activator.CreateInstance(assemblyType);
           else
              dynamicAssembly = Activator.CreateInstance(assemblyType, constructorParams);
       }
    }
    catch (Exception exc)
    {
       throw;
    }
    return dynamicAssembly;
}

When you want to initialize SMO object you have to know whole namespace, DLL name where object is implemented and constructor parameters. About method GetAssemblyPath check this solution.

Sample:

 var server = Load("Microsoft.SqlServer.Smo", "Microsoft.SqlServer.Management.Smo.Server", DataSource.ServerName);

There are a lot of pros and cons about using dynamic. More info you can find out on MS site http://msdn.microsoft.com/en-us/library/vstudio/dd264741.aspx

With this solution you do not have to bother which SQL server is installed on customer machine, because always is loaded the latest version from GAC.

The main disadvantage from programming view is that compiler can not know which type is used for dynamic (compile-time) and if you are not careful enough you can quickly produce (runtime) errors which can be hard to maintain.

About your other questions others already answered.

Regards,

Community
  • 1
  • 1
Miro Malek
  • 269
  • 2
  • 7
1

Here's some code that might help. I use it for a build/deployment system.

public void ExecuteScript(string scriptPath, SqlTransaction trans)
{
    var batch = new StringBuilder();
    var script = File.ReadAllLines(scriptPath);

    for (int i = 0; i < script.Length; i++)
    {
        if (script[i].Trim().StartsWith("GO", StringComparison.CurrentCultureIgnoreCase))
        {
            // If a line starts with a GO it means that a batch should be executed (Sql Server doesn't understand GO).
            ExecuteBatch(batch.ToString(), trans);
            batch = new StringBuilder();
        }
        else
            batch.AppendLine(script[i]);
    }
    // make sure we execute the last batch (it might not end with GO).
    ExecuteBatch(batch.ToString(), trans);        
}

private static void ExecuteBatch(string batch, SqlTransaction trans)
{
    batch = batch.Trim();
    if (batch == "") return;
    var cmd = new SqlCommand(batch, trans.Connection, trans);
    cmd.CommandTimeout = 0;
    cmd.ExecuteNonQuery();
}

NOTE: This isn't exactly the original code (I had some extra things in there that you probably don't care about) and I didn't test my changes above.

Brian
  • 37,399
  • 24
  • 94
  • 109
  • I was going to write in my question not to suggest that, since I'd seen that on various forums and dismissed it as "hack it till it works" logic. But [I gave it another look](http://msdn.microsoft.com/en-us/library/ms188037.aspx) after your suggestion here. It appears `GO` really is for batching, and this code theoretically should work, assuming `GO` isn't the only blocking factor on the scripts. I'll give it a shot :) Thanks! – Merlyn Morgan-Graham Nov 09 '11 at 23:53
  • +1; This was definitely helpful, and would have gotten me sorted out. Russel's code ended up being more directly useful, though it uses the same technique. I'm going to accept his answer since it was more directly usable, and add my additions to his code to my question. – Merlyn Morgan-Graham Nov 10 '11 at 01:21
  • No problem. His code does look a lot simpler than mine. In my production code I have a few other things I do with the sql (such as insert the contents of other files, like stored procedures, etc) which is why I wrote it the way I did. You might consider using a transaction in case there is an error in one of the scripts you can roll the changes back. – Brian Nov 10 '11 at 01:29
  • I'm blowing away the DB with this tool. It is *not* a tool for migration scripts, which we will have a totally different set up for. This is a quick-and-dirty tool for getting an EF program off the ground when there is no DB in production yet, and we don't have the Eye of Sauron errr our DBA looking at it yet :) Otherwise yeah, you're right about the transaction. – Merlyn Morgan-Graham Nov 10 '11 at 01:31