1

So I have been creating a library that uses dapper and allows user to manipulate a database.

I need some help with finding the best way to achieve the following.

Lets say I have an "order" table and I have a "transaction" table and an "order_line" table. I want to take the Increment Id of table "order" when inserting and use it to store it in a column in "transaction" and "order_line" table and I want all of this done in a SQL transaction so that I can roll back in case of any issue.

Now since my library is dynamic to any type and action, I am not sure on how to approach something like this.

Here is the code on how you would insert: I have 2 global variables

  private string connectionString { get; set; }

        public void newConnection(string connection)
        {
            if (string.IsNullOrWhiteSpace(connectionString))
            {
                connectionString = connection;
            }
        }

        private List<KeyValuePair<string, object>> transactions = new List<KeyValuePair<string, object>>();

Here is how you call to have a class to be saved to the database:

public void Add(object item)
{
    string propertyNames = "";
    string propertyParamaters = "";
    Type itemType = item.GetType();

    System.Reflection.PropertyInfo[] properties = itemType.GetProperties();

    for (int I = 0; I < properties.Count(); I++)
    {
        if (properties[I].Name.Equals("Id", StringComparison.CurrentCultureIgnoreCase) || properties[I].Name.Equals("AutoId", StringComparison.CurrentCultureIgnoreCase))
        {
            continue;
        }

        if (I == properties.Count() - 1)
        {
            propertyNames += "[" + properties[I].Name + "]";
            propertyParamaters += "@" + properties[I].Name;
        }
        else
        {
            propertyNames += "[" + properties[I].Name + "],";
            propertyParamaters += "@" + properties[I].Name + ",";
        }
    }

    string itemName = itemType.Name;
    KeyValuePair<string, object> command = new KeyValuePair<string, object>($"Insert Into[{ itemName}] ({ propertyNames}) Values({ propertyParamaters})", item);
    transactions.Add(command);
}

There are more methods and like edit, remove, edit list, remove list etc. but are not relevant in this case.

When you want to commit changes to the database you call:

public void SaveChanges()
        {
            using (SqlConnection sqlConnection = new SqlConnection(connectionString))
            {
                sqlConnection.Open();
                using (SqlTransaction sqlTransaction = sqlConnection.BeginTransaction())
                {
                    try
                    {
                        foreach (KeyValuePair<string, object> command in transactions)
                        {
                            sqlConnection.Execute(command.Key, command.Value, sqlTransaction);
                        }

                        sqlTransaction.Commit();
                    }
                    catch
                    {
                        sqlTransaction.Rollback();
                        throw;
                    }
                    finally
                    {
                        sqlConnection.Close();
                        transactions.Clear();
                    }
                }
                sqlConnection.Close();
            }

            transactions.Clear();
        }

You can find my library at github.com
https://github.com/pietercdevries/Bamboo.Net

Pieter de Vries
  • 825
  • 7
  • 18
  • 1
    I don't think that code compiles. I can't find any `Execute` method for `SqlConnection` in MSDN. – Barry O'Kane Mar 08 '17 at 07:49
  • I am using a library called dapper.net which is being used by stackoverflow actually. It just makes it easy to map object to a database. It is considered a micro orm. Like a mini version of entity framework. This should not matter in the scope of my question. – Pieter de Vries Mar 08 '17 at 08:09
  • I use storprocs and what you are looking for are output parameters, i do not know a lot about dapper but a quick google led me to this: http://stackoverflow.com/questions/28652987/alternative-way-to-get-output-parameter-from-stored-procedure The answer has an example on how to use output parameters in dapper – SilentStorm Mar 08 '17 at 08:28

2 Answers2

1

Can it be done... yes... should we be trying to do this ourselves... I wouldn't :) but lets try it any way.

Some ideas that can make this code simpler:

  • Define helper interfaces and force the data classes to implement them or use attribute declarations to specify id fields and foreign key references
  • Investigate Injection or code generation techniques so that you can get some of this 'dynamic' coding and lookup executed at compile time, not runtime.

I don't use Dapper and your SqlConnection.Execute() is an extension method I am not familiar with but I assume that it generates DbParameters from the passed in object and applies them to the SqlCommand when it gets executed. Hopefully dapper has some functions to extract the parameters, so that they can be used in this code example, or perhaps you can use some of these concepts and adapt them to your dapper code. I just want to acknowledge that upfront and that I have omitted any code example here that parameterises the objects when executing the commands.

This is the journey that the following snippets will go down

  1. Prepare the generated SQL to capture the Id field
  2. Output the Id value when we save changes
  3. Iterate over all remaining objects in the array and set the foreign key values

Note: these code changes are not tested or exception handled for production, nor would I call this "best practice" its just to prove the concept and help out a fellow coder :)

  1. You have already devised a convention for Id field tracking, lets extend that idea by preparing the sql statement to set the value of an output parameter:

    NOTE: in MS SQL, please use SCOPE_IDENTITY() in preference to @@Identity.
    What is the difference between Scope_Identity(), Identity(), @@Identity, and Ident_Current?

    NOTE: because the generated statements are using parameters, and we are not yet reading the parameter values, we will not need to regenerate the saved SQL statements later after we have found an Id value to insert to other objects... phew...

    public void Add(object item)
    {
        List<string> propertyNames = new List<string>();
        Type itemType = item.GetType();
    
        System.Reflection.PropertyInfo[] properties = itemType.GetProperties();
    
        for (int I = 0; I < properties.Count(); I++)
        {
            if (properties[I].Name.Equals("Id", StringComparison.CurrentCultureIgnoreCase) || properties[I].Name.Equals("AutoId", StringComparison.CurrentCultureIgnoreCase))
            {
                continue;
            }
            propertyNames.Add(properties[I].Name);
        }
    
        string itemName = itemType.Name;
        KeyValuePair<string, object> command = new KeyValuePair<string, object>
            ($"Insert Into[{itemName}] ({String.Join(",", propertyNames.Select(p => $"[{p}]"))}) Values({String.Join(",", propertyNames.Select(p => $"@{p}"))}); SET @OutId = SCOPE_IDENTITY();", item);
        transactions.Add(command);
        // Simply append your statement with a set command on an @id parameter we will add in SaveChanges()
    }
    
  2. In Save Changes, implement output parameter to capture the created Id, and if the Id was captured, save it back into the object that the command is associated to.

    NOTE: this code snippet shows the references to the solution in item 3. And the foreach was replaced with a for so we could do forward iterations from the current index

    public void SaveChanges()
    {
        using (SqlConnection sqlConnection = new SqlConnection(connectionString))
        {
            sqlConnection.Open();
            using (SqlTransaction sqlTransaction = sqlConnection.BeginTransaction())
            {
                try
                {
                    for (int i = 0; i < transactions.Count; i++)
                    {
                        KeyValuePair<string, object> command = transactions[i];
                        // 1. Execute the command, but use an output parameter to capture the generated id
                        var cmd = sqlConnection.CreateCommand();
                        cmd.Transaction = sqlTransaction;
                        cmd.CommandText = command.Key;
                        SqlParameter p = new SqlParameter()
                        {
                            ParameterName = "@OutId",
                            Size = 4,
                            Direction = ParameterDirection.Output
                        };
                        cmd.Parameters.Add(p);
                        cmd.ExecuteNonQuery();
    
                        // Check if the value was set, non insert operations wil not set this parameter
                        // Could optimise by not preparing for the parameter at all if this is not an 
                        // insert operation.
                        if (p.Value != DBNull.Value)
                        {
                            int idOut = (int)p.Value;
    
                            // 2. Stuff the value of Id back into the Id field.
                            string foreignKeyName = null;
                            SetIdValue(command.Value, idOut, out foreignKeyName);
                            // 3. Update foreign keys, but only in commands that we haven't execcuted yet
                            UpdateForeignKeys(foreignKeyName, idOut, transactions.Skip(i + 1));
                        }
                    }
    
                    sqlTransaction.Commit();
                }
                catch
                {
                    sqlTransaction.Rollback();
                    throw;
                }
                finally
                {
                    sqlConnection.Close();
                    transactions.Clear();
                }
            }
            sqlConnection.Close();
        }
    
        transactions.Clear();
    }
    
    
    /// <summary>
    /// Update the Id field of the specified object with the provided value
    /// </summary>
    /// <param name="item">Object that we want to set the Id for</param>
    /// <param name="idValue">Value of the Id that we want to push into the item</param>
    /// <param name="foreignKeyName">Name of the expected foreign key fields</param>
    private void SetIdValue(object item, int idValue, out string foreignKeyName)
    {
        // NOTE: There are better ways of doing this, including using interfaces to define the key field expectations.
        // This logic is consistant with existing code so that you are familiar with the concepts
        Type itemType = item.GetType();
        foreignKeyName = null;
    
        System.Reflection.PropertyInfo[] properties = itemType.GetProperties();
    
        for (int I = 0; I < properties.Count(); I++)
        {
            if (properties[I].Name.Equals("Id", StringComparison.CurrentCultureIgnoreCase) || properties[I].Name.Equals("AutoId", StringComparison.CurrentCultureIgnoreCase))
            {
                properties[I].SetValue(item, idValue);
                foreignKeyName = $"{item.GetType().Name}_{properties[I].Name}";
                break;
            }
        }
    }
    
  3. So now your objects have their Id's updated as they are inserted. Now for the fun part... After updating the Id, you should now iterate through the other objects and update their foreign key fields.

How you go about this in reality depends a lot on what kind of assumptions/conventions you are ready enforce over the data that you are updating. For simplicity sake, lets say that all of the foreign keys that we need to update are named with the convention {ParentClassName}_{Id}.

That means that if in our example we just inserted a new 'Widget', then we can try to forcibly update all other objects in this transaction scope that have a field 'Widget_Id' (or 'Widget_AutoId')

    private void UpdateForeignKeys(string foreignKeyName, int idValue, IEnumerable<KeyValuePair<string, object>> commands)
    {
        foreach(var command in commands)
        {
            Type itemType = command.Value.GetType();
            var keyProp = itemType.GetProperty(foreignKeyName);
            if(keyProp != null)
            {
                keyProp.SetValue(command.Value, idValue);
            }
        }
    }

This is a very simplistic example of how you could go about updating foreign (or reference) keys in OPs data persistence library. You have probably observed in reality that relational key fields are rarely consistently named using any convention, but even when conventions are followed, my simple convention would not support a table that had multiple references to parents of the same type, for example a Manifest in one of my client's apps has 3 links back to a user table:

public class Manifest
{
    ...
    Driver_UserId { get; set; }
    Sender_UserId { get; set; }
    Receiver_UserId { get; set; }
    ...
}

You would need to evolve some pretty advanced logic to tackle all possible linkage combinations.

Some ORMs do this by setting the values as negative numbers, and decrementing the numbers each type a new type is added to the command collection. Then after an insert you only need to update key fields that held the faked negative number with the updated number. You still need to know which fields are key fields, but atleast you don't need to track the precise fields that form the ends of each relationship, we can track with the values.

I like how Entity Framework goes about it though, try inject this linkage information about the fields using attributes on the properties. You may have to invent your own, but it's a clean declarative concept that forces you to describe these relationships up front in the data model classes in a way that all sorts of logic can later take advantage of, not just for generating SQL statements.

I don't want tobe too critical of Dapper, but once you start to go down this path or manually managing referential integrity like this there is a point where you should consider a more enterprise ready ORM like Entity Framework or nHibernate. Sure they come with some baggage but those ORMs have really evolved into mature products that have been optimised by the community. I now have very little manually written or scripted code to customise any interactions with the RDBMS at all, which means much less code to test or maintain. (= less bugs)

Community
  • 1
  • 1
Chris Schaller
  • 13,704
  • 3
  • 43
  • 81
  • Thanks. BTW Dapper is used and maintained by stackoverflow in case you where nor aware. Dapper is supper cool if you haven't used this, it saves quite some work. The reason for me to create this library is because entity framework and n-hibernate are loaded with baggage and I wanted to create one that has simple features like entity framework just not with all the baggage. It would just save time with simple operation and for more complex issues you would use dapper or ado.net – Pieter de Vries Mar 09 '17 at 21:55
  • yeah sorry about being a bit negative on dapper, it was pretty late. (i'll edit the post and tone it down) My point was kind of that once you start doing these sorts of things in the way that you are describing, then now you either end up with as much baggage of these other ORMs or worse, you end up being less performant. Most of the baggage is in maintaining referential integrity and supporting complex batch scenarios. After doing this for 15 years, when I see the amount of code you have written and now have to maintain, there is a point where your baggage is more expensive than theirs. – Chris Schaller Mar 09 '17 at 23:41
0

It doesn't say which database you are using. If it is MSSQL you can do

var id =  connection.Query<int?>("SELECT @@IDENTITY").SingleOrDefault();

after executing the Insert. That gives you the id of the last insert.

Palle Due
  • 5,929
  • 4
  • 17
  • 32
  • Yes that code is indeed correct. If you read my question than you realize that is not what I am after. Since I don't know the table even column yet I need to gather this. My question is if you where using my library how would you achieve the scenario I posted above? – Pieter de Vries Mar 08 '17 at 17:37