Create Dictionary with LINQ and avoid "item with the same key has already been added" error

Question

I want to find a key in a dictionary and replace the value if it is found or add the key/value if it is not.

Code:

public class MyObject
{

    public string UniqueKey { get; set; }
    public string Field1 { get; set; }
    public string Field2 { get; set; }

}

LINQ Solution (throws An item with the same key has already been added.):

Dictionary<string, MyObject> objectDict = csvEntries.ToDictionary(csvEntry => csvEntry.ToMyObject().UniqueKey, csvEntry => csvEntry.ToMyObject());

ForEach solution (works):

Dictionary<string, MyObject> objectDict = new Dictionary<string, MyObject>();
foreach (CSVEntry csvEntry in csvEntries)
{

    MyObject obj = csvEntry.ToMyObject();

    if (objectDict.ContainsKey(obj.UniqueKey))
    {
        objectDict[obj.UniqueKey] = obj;
    }
    else {
        objectDict.Add(obj.UniqueKey, obj);
    }

}

I really liked the LINQ solution but as it stands, it throws the above error. Is there any nice way of avoiding the error and using LINQ?

I would create a class `MyObjectComparer` that implements `IEqualityComparer` and use the following line: `Dictionary objectDict = csvEntries.Select(entry => entry.ToMyObject()).Distinct(new MyObjectComparer()).ToDictionary(csvEntry => csvEntry.UniqueKey, csvEntry => csvEntry);` — Jurgen Camilleri, Jul 20 '15 at 15:06
The indexer adds the element if it exists so you can just remove the `ContainsKey` check and always use `objectDict[obj.UniqueKey] = obj`. — Lee, Jul 20 '15 at 15:08
Your dictionary is a bit unusual. Every entry contains two copies of UniqueKey, one in the key and one in the object. — Graham, Jul 20 '15 at 15:28
For an efficient solution, see https://stackoverflow.com/a/22508992. — nawfal, Apr 20 '22 at 00:47

Tim Schmelter · Accepted Answer · 2022-10-20T13:19:11.980

25

You can use GroupBy to create unique keys:

Dictionary<string, MyObject> objectDict = csvEntries
    .Select(csvEntry => csvEntry.ToMyObject())
    .GroupBy(x => x.UniqueKey)
    .ToDictionary(grp => grp.Key, grp => grp.First());

However, instead of grp.First() you could create a collection with ToList or ToArray. On that way you don't take an arbitary object in case of duplicate keys. Or add your priority-logic in an OrderBy before First: grp => grp.OrderBy(x => Field1).ThenBy(x => x.Field2).First()

Another option is to use a Lookup<TKey, TValue> which allows duplicate keys and even non-existing keys, you get an empty sequence in that case.

var uniqueKeyLookup = csvEntries
    .Select(csvEntry => csvEntry.ToMyObject())
    .ToLookup(x => x.UniqueKey);
IEnumerable<MyObject> objectsFor1234 = uniqueKeyLookup["1234"]; // empty if it doesn't exist

edited Oct 20 '22 at 13:19

answered Jul 20 '15 at 15:10

Tim Schmelter

450,073
74
686
939

4

The first solution worked for me, however `ToDictionary(grp => grp.Key, grp => grp.First());` needs to be `ToDictionary(grp => grp.Key, grp => grp.Last());` because I want the key's value to be *replaced* by a later entry. – TomSelleck Jul 20 '15 at 16:31
@Tim Schmelter , I am facing similar issue with this statement- If TempDT IsNot Nothing Then m_ConcurrentScriptDictionary = TempDT.AsEnumerable.ToDictionary(Function(x) x.SafeField(fldClusterId, NULL_ID_VALUE), Function(y) y.SafeField(fldParamValue11, NULL_ID_VALUE)) , Can you help? – RSB Jan 03 '20 at 11:57

score 2 · Answer 2 · edited Aug 14 '21 at 10:02

Building on Tim's answer, here's an extension method you can use so you don't need to duplicate the implementation throughout your project:

public static class DictionaryExtensions
{
    public static Dictionary<TKey, TValue> ToDictionaryWithDupSelector<TKey, TValue>(
        this IEnumerable<TValue> enumerable,
        Func<TValue, TKey> groupBy, Func<IEnumerable<TValue>, TValue> selector = null) {

        if (selector == null)
            selector = new Func<IEnumerable<TValue>, TValue>(grp => grp.First());

        return enumerable
            .GroupBy(e => groupBy(e))
            .ToDictionary(grp => grp.Key, grp => selector(grp));
    }
}

By default it will choose the first element when there are duplicates, but I have provided an optional parameter where you can specify an alternate selector. Example call to the extension method:

var objList = new List<string[]> {
    new string[2] {"1", "first"},
    new string[2] {"1", "last"},
    new string[2] {"2", "you"},
};
var asDict = objList.ToDictionary(
    arr => arr[0],
    grp => grp.Last()
);

I am facing similar issue with this statement- If TempDT IsNot Nothing Then m_ConcurrentScriptDictionary = TempDT.AsEnumerable.ToDictionary(Function(x) x.SafeField(fldClusterId, NULL_ID_VALUE), Function(y) y.SafeField(fldParamValue11, NULL_ID_VALUE)) — RSB, Jan 03 '20 at 10:24

Create Dictionary with LINQ and avoid "item with the same key has already been added" error

2 Answers2