Mongoose, remove does not work inside aggregate

Question

Using Mongoose 4.2.9 with MongoDB 3 and the following data set:

Database: test
Collections: images, albums

I am trying to remove the images that belongs to no albums.

Sample document for albums collections:

> db.albums.findOne()
{
    "_id" : 0,
    "images" : [ 
        2433,
        2753,
        2983,
        6510,
        99334
    ]
}

Here images is an array of _id's from images collection.

Sample data for images collection:

> db.images.findOne()
{
    "_id" : 243,
    "height" : 480,
    "width" : 640
}

I have written the following code using node to achieve the mentioned goal:

var mongoose = require('mongoose');

/* Construct Album model */
var albumSchema = mongoose.Schema({
    _id: Number,
    images: [Number]
});

var Album = mongoose.model('Album', albumSchema);

/* Construct Image model */
var imageSchema = mongoose.Schema({
    _id: Number
});

var Image = mongoose.model('Image', imageSchema);

mongoose.connect('mongodb://localhost/test');
var db = mongoose.connection;
db.on('error', console.error.bind(console, 'connection error'));
db.once('open', doTheJob);

/* Do the job */

function doTheJob () { 
    Image.find({}, function (err, images) {
        if (err) {
            throw err;
        }
        for(var i in images){
            (function (image) {
                Album.aggregate([
                    {
                        $unwind: "$images"
                    }, {
                        $group: {
                            _id: "$images"
                        }
                    }, {
                        $match: {
                            _id: image._id
                        }
                    }
                ])
                .exec(function (err, albums) {
                    if (err) {
                        throw err;
                    }

                    if (albums.length === 0) {
                        console.log(image._id);
                        Image.remove({_id: image._id}, function(err){
                            if (err) {
                                console.log(err);
                                throw err;
                            }
                            console.log("Removed " + image._id);
                        });

                        /* Also the following code does not work, image (with 
                         * lowercase 'i' is a doc that is returned by Image.find({}))                              
                        image.remove( function(err){
                            if (err) {
                                console.log(err);
                                throw err;
                            }
                            console.log("Removed " + image._id);
                        });
                        */

                    }
                });
            })(images[i]);
        }
    });
}

Now my problem is Image.remove() is not called, since in console, I cannot see the error nor Removed XXX, also I tested the docs in console too.

I need to mention, if in the same program, instead of doTheJob(), I pass a callback with only a Image.remove({_id: XXX}, function () {}), then it works.

Hello, did you check the output of `console.log(albums)` when adding it immediate before of `if (albums.length === 0)`? — efkan, Dec 06 '15 at 09:47
@user3765109 Yes, I did, as a matter of fact, I got the problem, just have not got time to post it as an answer. Since the images collection was so large, 100000, it took more than 1 hour for the program to run, and surprisingly the `remove()` operations started firing after the 40 min. Do you have any idea why? — Musa Haidari, Dec 06 '15 at 09:50
Number of MongoDB Queries: 100,000 for albums + 100,000 * 5 for albums.images[] = 600,000 queries. Do collections and arrays have appropriate indexes? And do you have strong technical infrastructure? Actually if your application works good and doTheJob function is worked for the maintenance and rarely, perhaps we can say everything is OK. — efkan, Dec 06 '15 at 11:33
@user3765109 thank you for your information, also I mention that each Album contains a lot more images, like 100, that makes it more clear — Musa Haidari, Dec 06 '15 at 11:36
You might check this (http://stackoverflow.com/a/27854618/3765109) out when you are available. Good luck.. — efkan, Dec 06 '15 at 11:52

score 0 · Accepted Answer · answered Dec 06 '15 at 10:17

I got the problem to be the order in which mongoose executes callbacks.

Prior to execution of the program, images collection had 100,000 documents. When I run the program, it took around 1 hour to complete, during which, remove()'s started getting called/ fired after 40 min and that was the converging point, making me feel they are never called.

Mongoose, remove does not work inside aggregate

1 Answers1