MongoDB Unique Compound Index is Allowing Duplicates

Question

I'm using PyMongo and PyMODM to work with a fairly simple document structure.

This is what my model is like:

class User(MongoModel):
    subscriber_uid: fields.CharField = fields.CharField(required=True)
    interface: fields.CharField = fields.CharField(required=True)
    ...other fields...

    class Meta:
        """Defines MongoDB options for this model"""
        cascade = True  # currently nothing to cascade deletes against
        indexes = [
            IndexModel(
                keys=[('subscriber_uid', pymongo.ASCENDING),
                      ('interface', pymongo.ASCENDING)],
                unique=True,
            ),
        ]

My unit test looks like this, note that DB is an API which wraps the actual PyMongo commands. eg: put_user wraps user.save() in some exception handling logic, while read_user wraps User.objects.get().

...
    user = User(**TEST_USER)
    user.interface = '2/2/2/2'
    user.subscriber_uid = 'some other suid'
    DB.put_user(user)
    user_from_db = DB.read_user(subscriber_uid=user.subscriber_uid,
                                interface=user.interface)
    assert user_from_db.interface == user.interface
    assert user_from_db.subscriber_uid == user.subscriber_uid

    # attempt to create a new record with a non-unique suid+iface pair
    # ie: ensure this updates instead of creates
    user = User(**TEST_USER)
    user.someotherattr = 1023
    DB.put_user(user)
    user_from_db: User = DB.read_user(subscriber_uid=user.subscriber_uid,
                                      interface=user.interface)
    assert user_from_db.seed_index == user.seed_index
...

When I run this test, the read_user()/User.objects.get() call fails with a MultipleObjectsReturned objects returned error and I can confirm that there are two records with identical interface and subscriber_uid values.

From what I've seen in other StackOverflow answers and comments, the compound unique index above should prevent that from happening.

The database is being dropped on every test run, so the problem isn't stale data lingering. Am I misunderstanding the compound index?

score 1 · Accepted Answer · answered May 01 '20 at 23:44

The apparent lack of index was a red herring.

PyMODM docs state:

Note that connect() has to be called (defining the respective connection alias, if any) before any MongoModel can be used with that alias. If indexes is defined on Meta, then this has to be before the MongoModel class is evaluated.

I was using a function to initiate the connection to MongoDB, which of course came after I imported the model classes. With other database clients, this would typically be the way to go if you're unsure whether your DB will be available when the module is imported (common with containerized workloads, app startup usually takes less time than DB startup). The PyMongo client is special though:

Starting with version 3.0 the MongoClient constructor no longer blocks while connecting to the server or servers, and it no longer raises ConnectionFailure if they are unavailable, nor ConfigurationError if the user’s credentials are wrong. Instead, the constructor returns immediately and launches the connection process on background threads.

Moving the connect() to the top of my models module fixed the issue. Even though the DB isn't available when connect() is called. As soon as the database becomes available, the index gets created and all becomes right in the world.

MongoDB Unique Compound Index is Allowing Duplicates

1 Answers1