0

Situation: I have this method for inserting a list into one of my SQL Server table:

public void InsertDataAtOnce(List<string> values, List<string> millisecs, string table, string[] parameters)
{
        connection = new SqlConnection(connectionString);
        query = $"INSERT INTO {table} ({parameters[0]}, {parameters[1]}) VALUES (@{parameters[0]}, @{parameters[1]})";

        try
        {
            connection.Open();

            command = new SqlCommand(query, connection);
            command.Parameters.Add($"@{parameters[0]}", SqlDbType.NVarChar);
            command.Parameters.Add($"@{parameters[1]}", SqlDbType.NVarChar);

            for (int i = 0; i <millisecs.Count-2; i++)
            {
                //command.Parameters.Clear();
                command.Parameters[$"@{parameters[0]}"].Value = millisecs[i];
                command.Parameters[$"@{parameters[1]}"].Value = values[i];

                command.ExecuteNonQuery();
            }

            command.Dispose();
            connection.Close();
        }
        catch (Exception ex)
        {
            Console.WriteLine("Connection Error");
            Console.WriteLine(ex);
        }
}

Imagine my inserted table information should be like this (my table should hold around 1700 values) and I pass this information tho the method:

 1. 0.5 15
 2. 0.6 24
 3. 0.7 29
 4. 0.8 32
 5. 0.9 45

However, when I open the SQL Server Management Studio and check the inserted data, it mixes up the value order, so with this data, the result would be:

 1. 0.5 15
 2. 0.6 24
 3. 0.9 45
 4. 0.8 32
 5. 0.7 29

And so on. When I have my data, it mixes the data up in chunks, so the first few hundred values would be okay, then it jumps to the values, that should be in the end or the middle, returns back to the first values for a while and so on.

So what could be the problem here? I checked the Lists that I'm passing and they seem to be in order I added the data, just how they should be.

marc_s
  • 732,580
  • 175
  • 1,330
  • 1,459
  • 3
    SQL Server does not guarantee any order without an `ORDER BY`. It's a common but incorrect assumption. If you want the data to appear in a certain order, you need to include a value in your row that indicates the order. – Nick.Mc Mar 24 '20 at 10:34
  • 2
    There is **no problem** - it's the defined behavior. Any RDBMS never has any *system-inherent* ordering - the **only way** to get ordering is by specifying an explicit `ORDER BY` on your `SELECT` statement. You need to **stop assuming** you'll get back the data in the same order as you insert it - **that's just NOT the case!** – marc_s Mar 24 '20 at 10:35
  • Just summarising some stamements below. I suggest you use this method https://stackoverflow.com/questions/3913371/sqlbulkcopy-from-a-list to insert your list in one go (rather than RBAR as you are doing here). Then you can utilise `IDENTITY` to autogenerate your ordering key. Then _both_ answers below can be correct. :) – Nick.Mc Mar 28 '20 at 01:53

2 Answers2

3

A SQL table represent unordered sets. There is no ordering. However, you can use an identity() column to preserve insertion order:

create table t (
    t_id int identity(1, 1) primary key,
    . . .
);

When you insert into the table, do not include t_id into the column list. It will be set automatically.

Then you can fetch the results "in order" by doing:

select t.*
from t
order by t_id;
Gordon Linoff
  • 1,242,037
  • 58
  • 646
  • 786
  • kemdeveloper is trying to insert using InsertDataAtOnce() from c# and ado.net. This answer (creating an identity field) will not guarantee what he wants to acheive in case he calls the InsertDataAtOnce() method asynchronous. – gwt Mar 24 '20 at 12:39
  • @Karamafrooz . . . This answer guarantees that the results are returned in insert order, based on the inserts into the database. I have no idea why you don't think this is the case. – Gordon Linoff Mar 24 '20 at 14:11
  • I didn't say your answer doesn't return records in the order they have been inserted. But I said it does not guarantee what the question asks for in case he calls the method asynchronously. – gwt Mar 24 '20 at 14:30
  • In other words if the method is called asynchronously and the values for Id field are autogenerated, then there is no guarantee that records will be inserted in the order that he intends. – gwt Mar 24 '20 at 14:47
  • Wouldn't `command.ExecuteNonQuery();` need to be explicitly coded as asynchronous in this case? Or is that a property of some parent class that we can't see in the code sample? – Nick.Mc Mar 28 '20 at 01:55
1

The physical order of values getting inserted into a database table is identified based on an index that is called Clustered Index.

If it's important to you to insert the values in a specific order, you can create a clustered index on a specific field (for example an integer field that could also be or be not your primary key) and then provide the clustered index value in your insert statements before inserting them to the table.

Let's assume you have added an integer field with the name Id and created a clustered index on that field. If you insert the current values as:

    Id   p1 p2
 1.  1  0.5 15
 2.  2  0.6 24
 3.  3  0.7 29
 4.  4  0.8 32
 5.  5  0.9 45

Then if you query the table with an order by clause on the Id field the results you receive will be exactly as you have inserted them:

    Id   p1 p2
 1.  1  0.5 15
 2.  2  0.6 24
 3.  3  0.7 29
 4.  4  0.8 32
 5.  5  0.9 45
gwt
  • 2,331
  • 4
  • 37
  • 59
  • 3
    Even with a clustered index, you **DO NOT** get back the data in any specific, defined order and they are **NOT** ordered by the clustered index by default. You **ONLY** get back the data ordered if you **explicitly define** an `ORDER BY` clause in your `SELECT` ... this response is highly misleading .... – marc_s Mar 24 '20 at 12:13
  • @marc_s Based on my answer he is providing the id value manually to support the order he wants the records to be inserted. So if he selects with order by Id how will the resultset be different? – gwt Mar 24 '20 at 12:49
  • Fair enough. If he provides the Id value manually it's defnitely correct. The Clustered Index however plays no part in this. – Nick.Mc Mar 24 '20 at 12:50
  • @Nick.McDermaid The really important part that the clustered index is playing is in the insert part because it helps him to insert the records in the order he wants. It also helps him to retrieve the records in the future based on the order he has intended to insert them previousely. So I assume the clustered index is playing two important roles here! – gwt Mar 24 '20 at 12:52
  • 1
    The order that records are returned is not guaranteed to be clustered index order. Most of the time it is but it’s not guaranteed. – Nick.Mc Mar 24 '20 at 12:54
  • @Nick.McDermaid Yes, thanks to marcs comment, I know that now. But we are talking about the new answer now which is ordering columns by Id field for select operations. – gwt Mar 24 '20 at 12:55
  • @Karamafrooz . . . Rather strange. You commend that my answer is incorrect. Then you change your answer to incorporate my answer. – Gordon Linoff Mar 24 '20 at 14:13
  • @GordonLinoff My answer is not the same as your answer and does not incorporate it. I changed it based on Marc's comment. The difference between my answer and your's is that I am suggesting him to generate the Ids manually which will not have the issue I have commented on your answer. I am also explaining him the clustered index and how to use it to achieve what he wants that could not be found in your answer. – gwt Mar 24 '20 at 14:34
  • But the clustered index does not guarantee the order of records returned so it doesn't really have relevance to the answer. The order records are inserted only has revelance if you are using `IDENTITY` to capture the order, and as you pointed out below, if this is run async, even that won't work. The only part of your answer that is relevant is the last sentence. – Nick.Mc Mar 28 '20 at 01:52