Pymongo UUID search not returning documents that definitely exist

Question

Trying to define a function in python that can search for a given UUID like so:

def getid(in_id):
    return list(CollectionVar.find({"_id":UUID(in_id)}))

And passing in a UUID. I can take a UUID I know exists from Studio 3T like so:

db.getCollection("CollectionName").find({"_id":UUID("5002aa11-eeb7-4e68-a121-dd51497d2572")})

And the above query returns precisely one document. That same UUID in the python query returns absolutely nothing. I can find documents on other (non UUID) fields easily enough, for example the following works fine on that same document from earlier:

def getname(fn,sn):
    return list(CollectionVar.find({"Firstname":re.compile(fn, re.IGNORECASE), "Surname":re.compile(sn, re.IGNORECASE)}))

This seems like a problem with the uuid.UUID class rather than a pymongo issue? Can anyone see the problem?

PyMongo Version 3.6.1

score 7 · Answer 1 · answered Feb 25 '20 at 22:50

The issue is that PyMongo uses a legacy method of encoding/decoding UUID values by default. You probably want to configure the PyMongo client to use the more modern, cross-language compatible "standard" UUID representation:

client = MongoClient(MONGODB_URI, uuidRepresentation="standard")

Now you should be able to query directly using Python uuid.UUID instances:

from uuid import UUID

items = client["item_database"]["items"].find_one({
    "uuid": UUID("187382af-1369-43e6-a0ba-d345886c986c")
})

score 5 · Answer 2 · answered Jul 11 '18 at 12:59

5

I've solved this. For anyone else who hits this issue the solution is below:

from bson.binary import Binary, UUID_SUBTYPE

def getcust(inid):
    newuuid=uuid.UUID(inid).bytes
    return list(DealershipConsumer.find({"_id": Binary(bytes(bytearray(newuuid)), UUID_SUBTYPE)}))

UUID_SUBTYPE needs to be set to whatever subtype of UUID you use - in my case it's 4.

answered Jul 11 '18 at 12:59

MB141

114
1
7

Note that you can (and should) extract the UUID version from the UUID bytes rather than assuming a fixed value. – StephenS Nov 03 '18 at 23:12
`bytes(bytearray(` doesn't seem useful – Kiruahxh Apr 12 '23 at 13:22

Er... · Answer 3 · 2019-07-23T12:12:41.173

You could specify the UUID type you're using when getting the db :

import bson
import pymongo

mongo_client = pymongo.MongoClient(mongo_uri, document_class=dict)
db = mongo_client.get_database(                              
    "my_db_name",                                                          
    bson.codec_options.CodecOptions(uuid_representation=bson.binary.UUID_SUBTYPE),
)

If you want to read more about Mongo best practices while using UUIDs, this article might help.

Also, here are the docs about codec_options

score 0 · Answer 4 · answered Jul 11 '18 at 11:33

I've scraped this from pymongo-2.8.1 in the bson.binary.UUIDLegacy class's docstring/comment, might be usefull

  >>> import uuid
  >>> from bson.binary import Binary, UUIDLegacy, UUID_SUBTYPE
  >>> my_uuid = uuid.uuid4()
  >>> coll = db.test
  >>> coll.uuid_subtype = UUID_SUBTYPE
  >>> coll.insert({'uuid': Binary(my_uuid.bytes, 3)})
  ObjectId('...')
  >>> coll.find({'uuid': my_uuid}).count()
  0
  >>> coll.find({'uuid': UUIDLegacy(my_uuid)}).count()
  1
  >>> coll.find({'uuid': UUIDLegacy(my_uuid)})[0]['uuid']
  UUID('...')
  >>>
  >>> # Convert from subtype 3 to subtype 4
  >>> doc = coll.find_one({'uuid': UUIDLegacy(my_uuid)})
  >>> coll.save(doc)
  ObjectId('...')
  >>> coll.find({'uuid': UUIDLegacy(my_uuid)}).count()
  0
  >>> coll.find({'uuid': {'$in': [UUIDLegacy(my_uuid), my_uuid]}}).count()
  1
  >>> coll.find_one({'uuid': my_uuid})['uuid']
  UUID('...')

score -2 · Answer 5 · answered Jul 11 '18 at 10:38

-2

You need to use ObjectId instead of UUID. Try this, it works for me:

from bson.objectid import ObjectId


def getid(in_id):
    return list(CollectionVar.find({"_id":ObjectId(in_id)}))

answered Jul 11 '18 at 10:38

Oleksandr Yarushevskyi

2,789
2
17
24

"InvalidId: '5002aa11-eeb7-4e68-a121-dd51497d2572' is not a valid ObjectId, it must be a 12-byte input or a 24-character hex string" is the error message when using bson.objectid's ObjectId class. Do I have the wrong one? – MB141 Jul 11 '18 at 11:09
@MichaelBrown are you sure that your "_id" is UUID? If I don't mistake it's generating automatically by mongo and looks like "507f191e810c19729de860ea". – Oleksandr Yarushevskyi Jul 11 '18 at 11:22
Can you look at this https://stackoverflow.com/questions/9694460/difference-between-id-and-id-fields-in-mongodb? – Oleksandr Yarushevskyi Jul 11 '18 at 11:23
from Studio 3T (where searching for UUID("5002aa11-eeb7-4e68-a121-dd51497d2572") works fine) - https://i.imgur.com/H1ixJxe.png – MB141 Jul 11 '18 at 12:18

Pymongo UUID search not returning documents that definitely exist

5 Answers5