How to fetch data in a single go from mongodb based on multiple filters?

Question

I am new to mongodb and aggregation framework.

We have a class UserMetaData and a list of UserMetaData. I need to fetch data according to the userMetaDataList that is passed to the method solve(). Currently I am iterating the list and one by one fetching the corresponding collection from the monogdb. Since the db calls are made for each element in the list, this becomes a highly expensive operation. Is there any way to fetch all the required data from mongodb in one shot(more like a bulk fetch operation).

mongodb - perform batch query the solution provided in this does not fulfill the requirements of the current scenario. Please help!!

This is how I am doing currently.

class UserMetaData{
    String userId;
    String vehicleId;
    String vehicleColour;
    String orderId;
}

public List<String> getOrderIds(List<UserMetaData> userMetaDataList) {
    List<String> orderIds = new ArrayList<>();
    for (UserMetaData userMetadata : userMetaDataList) {
        try {
            BasicDBObject matchDBObject = new BasicDBObject("user_id", new BasicDBObject("$eq", userMetadata.getUserId()));
            matchDBObject.append("vehicle_id", new BasicDBObject("$eq", userMetadata.getVehicleID()));
            matchDBObject.append("vehicle_colour", new BasicDBObject("$in", ImmutableSet.of("WHITE", "BLACK")));
            Document document = eventCollection.find(matchDBObject)
                    .projection(new BasicDBObject("order_id", "1"))
                    .first();
            orderIds.add(document.get("order_id").toString());

        } catch (Exception e) {
            log.info("Exception occurred while fetching order id for user_id: {} asset_id:{} - {}", metadata.getUserId(), metadata.getAssetID(), e);
        }
    }
    return ordersIds;
}

I want to fetch all the corresponding data in a single query. Requesting help.

score 0 · Answer 1 · answered Feb 24 '22 at 10:38

0

You can join all filters with $OR condition and fetch the full list at once ...

answered Feb 24 '22 at 10:38

R2D2

9,410
2
12
28

prasad_ · Answer 2 · 2022-02-24T11:59:49.533

I want to fetch all the corresponding data in a single query.

You can use this approach and perform the query as a single operation (avoids the for-loop).

Consider sample documents in the collection test:

{ "_id" : ObjectId("621762e2cda7c6394d557f37"), "userid" : 1, "name" : "ijk", "orderid" : "11" }
{ "_id" : ObjectId("621762efcda7c6394d557f38"), "userid" : 12, "name" : "abc", "orderid" : "99" }
{ "_id" : ObjectId("621762fccda7c6394d557f39"), "userid" : 13, "name" : "xyz", "orderid" : "100" }

The array of objects to filter:

var DOCS = [
  { userid: 12, name: "abc" },
  { userid: 13, name: "xyz" }
]

The query to filter by DOCS:

db.test.find( 
  {
    $expr: { 
      $in: [ { userid: "$userid", name: "$name" },  DOCS ]
    }
  },
  { 
    orderid: 1
  }
)

The output has documents with userids 12 and 13.

[ EDIT - ADD ]

This aggregation an improvement over the find:

db.test.aggregate([

  // This matches the 'userid' and 'name' fields with the input list 'DOCS'
  { 
      $match: { 
          $expr: { 
              $in: [ { userid: "$userid", name: "$name" },  DOCS ]
          }
      }
  },

  // The grouping will select only the first matching for the 'userid' and 'name'
  // (this is as per the question post's code: `.first()`)
  {
      $group: { 
          _id: { 
             userid: "$userid", 
             name: "$name" 
          }, 
          orderid: { 
              $first: "$orderid" 
          }
      }
  },

  // Remove the '_id' field
  // Now the result has just the 'orderid' field only
  { 
      $unset: "_id" 
  }
])

Thank you for your suggestion. I strongly believe that this should work. But the thing is that $expr is supported in mongo 3.6+. Could you please help with the way to achieve the same in lower version than mongo 3.6. (essentially without $expr). — coderrabit, Feb 25 '22 at 10:06
The `$expr` is introduced with MongoDB v 3.6. See the [v3.6 Release Notes](https://docs.mongodb.com/v3.6/release-notes/3.6/#new-query-operators). — prasad_, Feb 25 '22 at 10:16

How to fetch data in a single go from mongodb based on multiple filters?

2 Answers2