MongoDB: Using match with input document variables

Question

Why do I have to use this code: { $match: { $expr: { <aggregation expression> } } } to match a document using a document input variable as opposed to doing: { $match: { <query> } } ?

For example:

    $lookup: {
      from: "comments",
      let: { myvar: '$myInputDocVariable'},
      pipeline: [
        { $match:
            { $expr:
               { $and:
                  [
                    { $eq: [ "$varFromCommentDocument",  "$$myvar" ] },
                  ]
               }
            }
         },
        ],
      as: "returnedValue"
    }

The query above works fine but the query below does not work as expected. Why is this? Does this mean that if you are using input variables in a $lookup pipeline you have to use $expr? why is that?

    $lookup: {
      from: "comments",
      let: { myvar: '$myInputDocVariable'},
      pipeline: [
        { $match: { "$varFromCommentDocument", "$$myvar" } }
      ],
      as: "returnedValue"
    }

Answer 1

When you perform uncorrelated sub-queries for $lookup operator:

• If you need to compare parent collection's field within pipeline, MongoDB cannot apply the standard query syntax (field:value) for variable / Aggregation expressions. In this case, you need to use $expr operator.

Example:

{ $match:
  { $expr:
    { $and:[
      { $eq: [ "$varFromCommentDocument",  "$$myvar" ] },
    ]}
  }
}

• if it matches against "hard-coded" values, you don't need to use $expr operator.

Example:

$lookup: {
  from: "comments",
  pipeline: [
    { $match:{ 
      "key": "value",
      "key2": "value2"
    }}
  ],
  as: "returnedValue"
}

Answer 2

Does this mean that if you are using input variables in a $lookup pipeline you have to use $expr

Yes correct, by default in filters i.e; in filter part of .find() or in $match aggregation stage you can't use an existing field in the document.

If at all if you need to use existing field's value in your query filter then you need to use aggregation pipeline, So in order to use aggregation pipeline in .find() or in $match you need to wrap your filter query with $expr. Same way to access local variables got created using let of $lookup filter in $match needs to be wrapped by $expr.

Let's consider below example :

Sample Docs :

[
  {
    "key": 1,
    "value": 2
  },
  {
    "key": 2,
    "value": 4
  },
  {
    "key": 5,
    "value": 5
  }
]

Query :

 db.collection.find({ key: { $gt: 1 }, value: { $gt: 4 } })

    Or

 db.collection.aggregate([ { $match: { key: { $gt: 1 }, value: { $gt: 4 } } } ])

Test : mongoplayground

If you see the above query both input 1 & 4 are passed into query but it you check below query where you try to match key field == value field - it doesn't work :

db.collection.aggregate([ { $match: { key: { $eq: "$value" } } } ])

Test : mongoplayground

Above as you're comparing two existing fields then you can't do that as it mean you're checking for docs with key field value as string "$value". So to say it's not a string it's actually a reference to value field you need to use $eq aggregation operator rather than $eq query operator like below :

 db.collection.aggregate([ { $match: { $expr: { $eq: [ "$key", "$value" ] } } } ])

Test : mongoplayground