How to read data from Mongodb which have duplicate element name in c#

Question

I am using MongoDB.Drivers in my C# MVC application to communicate with Mongodb database.

C# Code

            var client = new MongoClient("mongodb://localhost:27012");
            var db = client.GetDatabase("Test_DB");
            var collection = db.GetCollection<BsonDocument>("TestTable");
            var tData = await collection.FindAsync(new BsonDocument(true)); // I used - new BsonDocument(true)  to specify allow duplicate element name while read data.

MongoDB Data Pic

In above image you can see I have multiple Columns called DuplicateCol with different values. When I tried to read these data in c# using MongoDB.Driver I got following error : InvalidOperationException: Duplicate element name 'DuplicateCol'.

While insert duplicate element name I used AllowDuplicateNames=true of BsonDocument object as below. (It insert duplicate element name without error.)

BsonDocument obj = new BsonDocument();
obj.AllowDuplicateNames = true;
obj.Add("DuplicateCol", "Value_One");
obj.Add("propone", "newVal");
obj.Add("DuplicateCol", "Value_Two");
.... // other properties with value 
await collection.InsertOneAsync(obj);

Note: This Schema is Must. I can not altered it.

Please provide me suggestions to fix this Issue. Any help would be highly appreciated..

Thanks.

A quick question, is this a must, or can the schema be altered at this point? — Nir, Feb 05 '18 at 11:39
@Nir - It is must. I need this kind of data in real scenario — prog1011, Feb 05 '18 at 11:41
I'm not sure how you managed to create such a document. In MongoDB document keys should be unique. Even if you manage to fix your current error, you will meet the problems here and there with such schema. — CodeFuller, Feb 06 '18 at 05:31
@CodeFuller - I used `AllowDuplicateNames=true` property of `BsonDocument` object. — prog1011, Feb 06 '18 at 11:21
Is that really necessary? Is it not possible to replace this by an array? Even the C# driver documentation considers this as "not recommended". — Iľja, Feb 14 '18 at 08:38

Evk · Accepted Answer · 2018-02-07T13:26:21.953

If nothing else helps, you reviewed other answers and comments, and still think you absolutely must keep design described in your question, you can use the following hack. Create class like this:

class AlwaysAllowDuplicateNamesBsonDocumentSerializer : BsonDocumentSerializer {
    protected override BsonDocument DeserializeValue(BsonDeserializationContext context, BsonDeserializationArgs args) {
        if (!context.AllowDuplicateElementNames)
            context = context.With(c => c.AllowDuplicateElementNames = true);
        return base.DeserializeValue(context, args);
    }

    public override BsonDocument Deserialize(BsonDeserializationContext context, BsonDeserializationArgs args) {
        if (!context.AllowDuplicateElementNames)
            context = context.With(c => c.AllowDuplicateElementNames = true);
        return base.Deserialize(context, args);
    }
}

This is custom serializer for BsonDocument which always sets AllowDuplicateElementNames while deserializing. Then you need a bit of reflection to overwrite default BsonDocument serializer (because BsonDocumentSerializer.Instance has no setter):

// get __instance field, which is backing field for Instance property
var instanceField = typeof(BsonDocumentSerializer).GetField("__instance", BindingFlags.Static | BindingFlags.NonPublic);
// overwrite with our custom serializer
instanceField.SetValue(null, new AlwaysAllowDuplicateNamesBsonDocumentSerializer());

By doing that somewhere at startup you will be able to read back your documents with duplicated attributes.

Full code for test:

static void Main(string[] args) {
    var instanceField = typeof(BsonDocumentSerializer).GetField("__instance", BindingFlags.Static | BindingFlags.NonPublic);
    instanceField.SetValue(null, new AlwaysAllowDuplicateNamesBsonDocumentSerializer());
    TestMongoQuery();
    Console.ReadKey();
}

static async void TestMongoQuery() {
    var client = new MongoClient();
    var db = client.GetDatabase("Test_DB");            
    var collection = db.GetCollection<BsonDocument>("TestTable");
    using (var allDocs = await collection.FindAsync(FilterDefinition<BsonDocument>.Empty)) {
        while (allDocs.MoveNext()) {
            foreach (var doc in allDocs.Current) {
                var duplicateElements = doc.Elements.Where(c => c.Name == "DuplicateCol");
                foreach (var el in duplicateElements) {
                    Console.WriteLine(el.Name + ":" + el.Value);
                }
            }
        }
    }
}

thanks for the reply. But can you please help me that What should I write instead of `await collection.FindAsync(new BsonDocument());` It seems that I am almost close to fix this issue ... — prog1011, Feb 07 '18 at 12:46
@prog1011 just query documents the way you usually do, for example `collection.FindAsync(FilterDefinition.Empty);` to get all documents, or `collection.FindAsync(new BsonDocument("x", 10))` to find all documents where x = 10. Duplicates will be allowed for any query with this hack. — Evk, Feb 07 '18 at 12:50
Still I am getting same error .. can you please tell me how can I use custom serializer for BsonDocument ? — prog1011, Feb 07 '18 at 13:00
@prog1011 updated answer with copy-pastable console example. — Evk, Feb 07 '18 at 13:05
But , When I read those data .. I always get first element for duplicate keys. Is there any way to get all duplicate data ? — prog1011, Feb 07 '18 at 13:21
@prog1011 elements are there, but you cannot use indexer (`doc["DuplicateCol"]`) to access them, because it is not designed for duplicate elements and will always return first. But you can access elements directly via `doc.Elements`, I've updated sample code in answer to show that. — Evk, Feb 07 '18 at 13:27
@prog1011 you can also create extension method for `BsonDocument` which will assume duplicate elements are possible and will always return array of `BsonValue` (so the same as indexer, but possibly returning multiple elements while indexer always returns 1). — Evk, Feb 07 '18 at 13:33

pieperu · Answer 2 · 2018-02-06T16:09:37.277

4

The error you receive is by design

InvalidOperationException: Duplicate element name 'DuplicateCol'

As CodeFuller said, MongoDB document keys should be unique. If you need a document to contain duplicate field names and you cannot alter this schema, MongoDB might not be the right database solution for you....I don't know which one will to be honest. Although it does seem possible to save duplicate keys using:

AllowDuplicateNames=true

I imagine you will experience challenges with querying and indexing, among others.

An argument could be made that this schema is a very strange requirement. A more appropriate schema might be:

{
    "_id" : ObjectId("xxx"),
    "propTest" : 0,
    ...
    "duplicateCol": [ "Value_Two", "Value_One" ]
}

Here you have a single property (duplicateCol) which is an array which accepts multiple strings.

edited Feb 06 '18 at 16:09

answered Feb 06 '18 at 11:28

pieperu

2,662
3
18
31

But, there has to be other way to read data. Because my schema is working fine while Insert(using `AllowDuplicateNames=true`), then there should be workaround to read these kind of data – prog1011 Feb 06 '18 at 11:45
There are a lot of considerations to make here. How will MongoDB handle indexing these duplicate document keys? When you query the document using most drivers i imagine they will fall over when querying using an Equality comparer, how would it know which of your duplicate field to compare against? This is a very unnatural schema you have. Is there any reason you cannot change your schema? I'll try to help but this is a very strange case. Can i ask the application you are using this in? Sensor readings? Form data? – pieperu Feb 06 '18 at 16:07
My application will read data from `Windows Active Directory Attributes` and there can be duplicate attributes with multiple values. So I need schema as I discussed in OP. I can not change that Schema. – prog1011 Feb 07 '18 at 06:04
Does your data schema have to mirror that of Windows Active Directory? Can you not save it in a more correct format in your database and format it into a different structure in your application itself. I'm afraid i do not know what to suggest in terms of storing duplicate document keys, other than to advise against it. – pieperu Feb 07 '18 at 09:46

How to read data from Mongodb which have duplicate element name in c#

2 Answers2