12

Our React app has a live search that is connected to a MongoDB collection with the names of 250K individuals. On key stroke, the Live Search uses regex to grab the top 20 documents in the collection. Our data fetching route for the Live Search is straightforward and looks like this:

router.get('/live-search/text/:text', function (req, res) {
    let text = req.params.text;
    let queryFilters = { label: { $regex: `${text}`, $options: 'i' } };

    db.gs__ptgc_selects.find(queryFilters).limit(20)
        .then(data => res.json(data))
        .catch(err => res.status(400).json('Error: ' + err))
});

I am certain that our collection has the text index, and it is set on the label column. Here is a screenshot of our MongoDB Atlas page showing as such:

enter image description here

The problem is that the index does not appear to be working. Whenever our app's live search is used (which is all the time), I continue to get email warnings for Query Targeting: Scanned Objects / Returned has gone above 1000. I've checked the MongoDB Atlas Profiler for these queries, and (I think) confirmed that the index is not being used:

enter image description here

Am I missing something? It's hard to create a fully reproducible example out of this. We've been struggling with the MongoDB text index and using it with our Live Search.

Edit

Maybe I need to be using $search, like it's done here: https://docs.mongodb.com/manual/text-search/. I am currently string matching using regex, but am not using mongo's $search.

halfer
  • 19,824
  • 17
  • 99
  • 186
Canovice
  • 9,012
  • 22
  • 93
  • 211
  • Based on [MongoDB regex query documentation](https://docs.mongodb.com/manual/reference/operator/query/regex/#index-use) regex query might not be compatible with the indexes available. – Attila Viniczai Oct 19 '20 at 19:50
  • 2
    Yes, if you only need to search for a character sequence, then `$search` is the way to go. Ref: https://stackoverflow.com/questions/35812680/searching-in-mongo-db-using-mongoose-regex-vs-text – Attila Viniczai Oct 19 '20 at 19:52
  • @AttilaViniczai how problematic would you say using $regex is? The live search with regex still works fast enough without taking advantage of text indexing. If hundreds of people are using this live search, should I be concerned about hundreds of inefficient scans of the database? – Canovice Oct 19 '20 at 20:11
  • I would turn this around and suggest making a decision based on requirements. So the question is, what is the search requirement of the `/live-search/text/` endpoint? If said requirement is similar to "match a sequence of characters", then go with `$search`. Else, if the requirement is something like "execute a regular expression search", then `$regex` could be needed. Also, performance of `$regex` queries depends on what kind of regular expression is used. Ref: https://stackoverflow.com/questions/17501798/mongodb-performance-of-query-by-regular-expression-on-indexed-fields – Attila Viniczai Oct 19 '20 at 20:24
  • 1
    Thanks for sharing. I spun this latter question into its own Stackoverflow question, since identifying that `$regex` does not work with the text index resolved this question. – Canovice Oct 19 '20 at 20:31
  • The regex will work fine until you have enough data that it exceeds what can be stored in the in-memory cache. After that the collection scans will get progressively worse due to thrashing the disk. – Joe Sep 28 '21 at 14:50

0 Answers0