Mongodb aggregation lookup with conditions

Question

I have a collection called article_category which store all article_id belongs to the category with category_id with data format like so.

Collection 1: article_category

{
  "article_id": 2015110920343902,
  "all_category_id": [5,8,10]
}

Then I have other collection called article which store all my post

Collection 2: article

{
  "title": "This is example rows in article collection"
  "article_id": 2015110920343902,
},
{
  "title": "Something change"
  "article_id": 2015110920343903,
},
{
  "title": "This is another rows",
  "article_id": 2015110920343904,
}

Now I want to perform MongoDB query to find title with regex while category_id must equal to 8. Here is my query but is not work.

db.article.aggregate(
{
  $match: 
  {
    title: 
    {
       $regex: /example/
    }
  }
},
{
    $lookup:
       {
         from: "article_category",
         pipeline: [
            { $match: { category_id: 8 } }
         ],
         as: "article_category"
       }
  }
)

Above query only show the records which match by regex but not match by category_id.

Any idea?

Alex Blex · Accepted Answer · 2018-01-30T11:08:48.247

First of all, it is all_category_id, not category_id. Secondly, you don't link articles - all documents will have exactly the same article_category array. Lastly, you probably want to filter out articles that don't have matched category. The conditional pipeline should look more like this:

db.article.aggregate([
  { $match: {
      title: { $regex: /example/ }
  } },
  { $lookup: {
    from: "article_category",
    let: {
      article_id: "$article_id"
    },
    pipeline: [
      { $match: {
          $expr: { $and: [
              { $in: [ 8, "$all_category_id" ] },
              { $eq: [ "$article_id", "$$article_id" ] }
          ] }
      } }
    ],
    as: "article_category"
  } },
  { $match: {
    $expr: { $gt: [
      { $size: "$article_category"},
      0
    ] }
  } }
] )

UPDATE:

If you don't match article_id, the $lookup will result with identical article_category array to all articles.

Let's say your article_category collection has another document:

{
  "article_id": 0,
  "all_category_id": [5,8,10]
}

With { $eq: [ "$article_id", "$$article_id" ] } in the pipeline the resulting article_category is

[ 
  { 
    "article_id" : 2015110920343902, 
    "all_category_id" : [ 5, 8, 10 ] 
  } 
]

without:

[ 
  { 
    "article_id" : 2015110920343902, 
    "all_category_id" : [ 5, 8, 10 ] 
  },
  {
    "article_id": 0,
    "all_category_id": [ 5, 8, 10 ]
  }
]

If the later is what you need, it would be way simpler to make to find requests:

db.article.find({ title: { $regex: /example/ } })

and

db.article_category.find({ all_category_id: 8 })

I don't understand why you compare `article_id`, I just need query where `title` is contains `sample` AND `article_category.all_category_id` is equal to `8` — Viet Nguyen, Jan 30 '18 at 10:42
@vietnguyen09 because `and` implies both should match. see the update. — Alex Blex, Jan 30 '18 at 11:09
Your query is perfect and works like a charm, but it took me 26s to get result. I've been adding the index for `article_id` and `all_category_id` but no luck. Could you suggest me how to make it faster? I have added limit 10 but it still took 3s for result — Viet Nguyen, Jan 31 '18 at 13:28
@vietnguyen09, Try Veeram's one. You don't really need conditional lookup here. I only kept it because you explicitly asked how to use conditional lookup in your case. I believe the regex is what takes most of the time, but to say for sure you need to provide output of `explain` command with `allPlansExecution` option. — Alex Blex, Jan 31 '18 at 14:03

s7vr · Answer 2 · 2018-01-30T10:52:42.170

You've couple of things incorrect here. category_id should be all_category_id. Use the join condition in $lookup and move the $match outside of $lookup stage with $unwind for optimized lookup.

Use $project with exclusion to drop the looked up field from final response. Something like {$project:{article_category:0}}

Try

db.article.aggregate([
  {"$match":{"title":{"$regex":/example/}}},
  {"$lookup":{
    "from":"article_category",
    "localField":"article_id",
    "foreignField":"article_id",
    "as":"article_category"
  }},
  {"$unwind":"$article_category"},
  {"$match":{"article_category.all_category_id":8}}
])

For uncorrelated subquery try

db.article.aggregate([
  {"$match":{"title":{"$regex":/example/}}},
  {"$lookup":{
    "from":"article_category",
    "pipeline":[{"$match":{"all_category_id":8}}],
    "as":"categories"
  }},
  {"$match":{"categories":{"$ne":[]}}}
])

Mongodb aggregation lookup with conditions

2 Answers2

Linked

Related