Spring data MongoDb Aggregation lookup cascading

Question

I'm using Spring (boot) 2.2.7 with mongoDB 4.0 running on Debian Stretch (9.12). I've setup 3 collections that I'm trying to join via an aggregation lookup cascading operation.

• operations (Node class)
• stock (Component class)
• catalog (Product class)

Every class has an uuid string property that links them together.

Node (sensors, sinks, base stations ... etc)

@Document(collection = "operations")
public class Node implements Serializable {
    @Id
    private String _id;
    private String componentUuId;
....
}

Component (a delivered product with serial number, barecode ...)

@Document(collection = "stock")
public class Component implements Serializable {
    @Id
    private String _id;
    private String uuId;
    private String productUuId;
....
}

Product (a market artefact)

@Document(collection = "catalog")
public class Product implements Serializable {
    @Id
    private String _id;
    private String uuId;
....
}

when running this command in the mongo shell :

db.operations.aggregate([
   {
     $lookup: {
         "from": "stock",
         "localField": "componentUuId",
         "foreignField": "uuId",
         "as": "component"
     }
   },
   {
     $lookup: {
         "from": "catalog",
         "localField": "component.productUuId",
         "foreignField": "uuId",
         "as": "product"
     }
   }
]).pretty()

I get

{
    "_id" : ObjectId("5ec952a7714e7e248b5fb98b"),
    "uuId" : "3b5e3417-166c-4d69-9fa5-b0856d819541",
    "componentUuId" : "57eff5c5-8b3c-43cb-9f2d-1c4098fc0c1f",
    "component" : [
        {
            "_id" : ObjectId("5ec952a4714e7e248b5fb97f"),
            "uuId" : "57eff5c5-8b3c-43cb-9f2d-1c4098fc0c1f",
            "productUuId" : "3da7a6a6-ccf6-4fe1-892a-a6d6b82396c1",
            "created" : ISODate("2020-05-23T14:43:16.588Z"),
            "_class" : "org.open_si.udc_common.models.Component"
            .......
        }
    ],
    "product" : [
        {
            "_id" : ObjectId("5ec952a1714e7e248b5fb969"),
            "uuId" : "3da7a6a6-ccf6-4fe1-892a-a6d6b82396c1",
            "model" : "TMP36GT9Z",
            "_class" : "org.open_si.udc_common.models.Product"
            ......
        }
    ]
}

so to reach the same goal in Java (w/ Spring-data-mongodb) I ran

Aggregation agg = Aggregation.newAggregation(
    Aggregation.lookup(mongoTemplate.getCollectionName(Component.class),
        "componentUuId", "uuId", "component"),
    Aggregation.lookup(mongoTemplate.getCollectionName(Product.class),
        "component.productUuId", "uuId", "product")
);

AggregationResults<NodeExpanded> results = mongoTemplate.aggregate(agg, mongoTemplate.getCollectionName(Node.class), NodeExpanded.class);

return results.getMappedResults();

using the NodeExpanded class

public class NodeExpanded implements Serializable {

    private String uuId;
    private Component component;
    private Product product;
....
}

note : I don't extend the Node base class as I get a duplicated key error before the following issue.

the problem is that I face an error in the service hosting that code :

Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is java.lang.NullPointerException] with root cause

java.lang.NullPointerException: null

unfortunately I don't know how to debug at lower level in Spring and can't find out what's the original error.

Note that the code bellow to get a Product linked to a Component works fine :

Aggregation agg = Aggregation.newAggregation(
                Aggregation.lookup(mongoTemplate.getCollectionName(Product.class),
                        "productUuId", "uuId", "product")
        );

AggregationResults<ComponentExpanded> results =
                mongoTemplate.aggregate(agg,mongoTemplate.getCollectionName(Component.class), ComponentExpanded.class);
return results.getMappedResults();

with the expanded class :

public class ComponentExpanded implements Serializable {
    private String uuId;
    private Product product;
...
}

any idea ? thanks in advance.

---

EDIT

trying to debug deeper, I've restricted the code to only join the operations collection(Node.class) to the stock one (Component.class) in the same way I've done with the stock (Component.class) and catalog (Product.class) that works fine.

public List<NodeExpanded> list() {

    Aggregation agg = Aggregation.newAggregation(
            Aggregation.lookup(mongoTemplate.getCollectionName(Component.class),
                    "componentUuId", "uuId", "component")

    );

    AggregationResults<NodeExpanded> results = mongoTemplate.aggregate(agg, mongoTemplate.getCollectionName(Node.class), NodeExpanded.class);
    return results.getMappedResults();
}

with a simple NodeExpanded class as

public class NodeExpanded implements Serializable {

    private String uuId;
    private Component component;

}

and I get the same null pointer exception on the mongoTemplate.aggregate() instruction

---

SOLVED

The problem comes form the SensorPersistenceService that doesn't @Autowired the MongoTemplate. With the informations given in that post it was not possible to find out the error cause. Thanks again to @Valijon for his help during this quest ^^

Answer 1

Probably the problem is NodeExpanded expects single Component and Product, but $lookup aggregation returns an array for both.

If you add extra $project stage, it should solve your problem.

Aggregation agg = Aggregation.newAggregation(
    Aggregation.lookup(mongoTemplate.getCollectionName(Component.class), 
        "componentUuId", "uuId", "component"),
    Aggregation.lookup(mongoTemplate.getCollectionName(Product.class), 
        "component.productUuId", "uuId", "product"),
    Aggregation.project()
        .andInclude("uuId") //put here all fields for NodeExpanded
        .and(ArrayOperators.ArrayElemAt.arrayOf("component").elementAt(0)).as("component")
        .and(ArrayOperators.ArrayElemAt.arrayOf("product").elementAt(0)).as("product")
);