Storing a Dictionary or KeyValuePair in a database

Question

I wanted to see what others have experienced when working with types like List<> or Dictionary<> and having in turn storing and retrieving that data?

Here's an example scenario: users will be creating their own "templates", where these templates is essentially a collection of Dictionary, e.g. for user1, values are (1, Account), (2, Bank), (3, Code), (4, Savings), and for user2, values (unrelated) could be (1, Name), (2, Grade), (3, Class), and so on. These templates/lists could be of varying length but they will always have an index and a value. Also, each list/ template will have one and only one User linked to it.

What types did you choose on the database side?

And pain-points and/or advice I should be aware of?

You mean a pain point other than the fact that List is not valid C# syntax? — David L, Nov 14 '14 at 03:29
`List` and `Dictionary` are extremely common types used daily by many C# developers. Without your specific concerns it will be impossible to give a good response to your question. — Jonesopolis, Nov 14 '14 at 03:30
There's some decent answers to this question that are applicable ... http://stackoverflow.com/questions/21052318/storing-a-dictionarystring-string-in-the-database — dodgy_coder, Nov 14 '14 at 03:33
I don't store .NET objects in SQL. I store the data needed to fill custom objects and then add them to .NET collections if necessary. — TyCobb, Nov 14 '14 at 03:37

score 1 · Answer 1 · answered Nov 14 '14 at 03:40

List or any collection's representation in databases are supposed to be tables. Always think of it as a collection and relate it to what a database offers.

Though you can always serialize a collection, i do not suggest it since updating or inserting records, you'd always update the whole record or data whereas having a table, you'd only have to query for the KEY wherein Dictionary, you already have it.

score 1 · Accepted Answer · edited May 23 '17 at 11:44

As far as the types within the collection go, there is a fairly 1-to-1 mapping between .Net types and SQL types: SQL Server Data Type Mappings. You mostly need to worry about string fields:

Will they always be ASCII values (0 - 255)? Then use VARCHAR. If they might contain non-ASCII / UCS-2 characters, then use NVARCHAR.
What is their likely max length?

Of course, sometimes you might want to use a slightly different numeric type in the database. The main reason would be if an int was chosen on the app side because it "easier" (or so I have been told) to deal with than Int16 and byte, but the values will never be above 32,767 or 255, then you should most likely use SMALLINT or TINYINT respectively. The difference between int and byte in terms of memory in the app layer might be minimal, but it does have an impact in terms of physical storage, especially as row counts increase. And if that is not clear, "impact" means slowing down queries and sometimes costing more money when you need to buy more SAN space. But, the reason I said to "most likely use SMALLINT or TINYINT" is because if you have Enterprise Edition and have Row Compression or Page Compression enabled, then the values will be stored in the smallest datatype that they will fit in.

As far as retrieving the data from the database, that is just a simple SELECT.

As far as storing that data (at least in terms of doing it efficiently), well, that is more interesting :). A nice way to transport a list of fields to SQL Server is to use Table-Valued Parameters (TVPs). These were introduced in SQL Server 2008. I have posted a code sample (C# and T-SQL) in this answer on a very similar question here: Pass Dictionary<string,int> to Stored Procedure T-SQL. There is another TVP example on that question (the accepted answer), but instead of using IEnumerable<SqlDataRecord>, it uses a DataTable which is an unnecessary copy of the collection.

EDIT: With regards to the recent update of the question that specifies the actual data being persisted, that should be stored in a table similar to:

UserID INT NOT NULL,
TemplateIndex INT NOT NULL,
TemplateValue VARCHAR(100) NOT NULL

The PRIMARY KEY should be (UserID, TemplateIndex) as that is a unique combination. There is no need (at least not with the given information) for an IDENTITY field.

The TemplateIndex and TemplateValue fields would get passed in the TVP as shown in my answer to the question that I linked above. The UserID would be sent by itself as a second SqlParameter. In the stored procedure, you would do something similar to:

INSERT INTO SchemaName.TableName (UserID, TemplateIndex, TemplateName)
    SELECT  @UserID,
            tmp.TemplateIndex,
            tmp.TemplateName
    FROM    @ImportTable tmp;

And just to have it stated explicitly, unless there is a very specific reason for doing so (which would need to include never, ever needing to use this data in any queries, such that this data is really just a document and no more usable in queries than a PDF or image), then you shouldn't serialize it to any format. Though if you were inclined to do so, XML is a better choice than JSON, at least for SQL Server, as there is built-in support for interacting with XML data in SQL Server but not so much for JSON.

Holy cow, I just took a look at your answer for the "Pass Dictionary..." and it does indeed look like it should help me! I'll report back and thanks! — emily_bma, Nov 14 '14 at 04:34
@frenu : I just updated my answer here with respect to the new details provided in your question update, including some variance of my linked answer that would be only slightly different here due to the UserID property / field. — Solomon Rutzky, Nov 14 '14 at 04:37
Let me take in this great information slowly and I'll report back. Thanks! — emily_bma, Nov 14 '14 at 04:49

Storing a Dictionary or KeyValuePair in a database

2 Answers2

Linked