mongoose.connection.collections.collection.drop() throws error every other time

Question

I am setting up testing using Jest for an Node/Express/Mongo project. I have tried to write a function to clear collections so each test starts with a clean slate:

const clearCollection = (collectionName, done) => {
  const collection = mongoose.connection.collections[collectionName]
  collection.drop(err => {
    if (err) throw new Error(err)
    else done()
  )
}

beforeEach(done => {
  clearCollection('users', done)
})

And another try, with promises:

const clearCollection = collectionName => {
  const collection = mongoose.connection.collections[collectionName]
  return collection.drop()
}

beforeEach(async () => {
  await clearCollection('users')
})

The problem is that it they both alternate between working and throwing an error. Every time I save the file, it either works perfectly, or throws an error, alternating each time. The errors are always either one of:

MongoError: cannot perform operation: a background operation is currently running for collection auth_test.users

MongoError: ns not found

I can get it to work 100% of the time (limited by the stack anyway) by making clearCollection() call itself inside a catch(), but this feels so wrong:

const clearCollection = collectionName => {
  const collection = mongoose.connection.collections[collectionName]
  return collection.drop()
    .catch(() => clearCollection(collectionName))
}

FWIW, most (if not all) async MongoDB methods return promises, so `return collection.drop()` should be sufficient. — robertklep, Mar 23 '17 at 06:42
You are right, but it still throws the same errors every other time. I'll update my question to reflect your suggestion. — neurozero, Mar 23 '17 at 06:47
It smells a bit like the promise being resolved before the drop operation being completed (AFAICS, dropping a collection locks the database, which would explain the first error you're getting). I doubt it'll fix anything, but have you tried making `beforeEach()` return a promise instead of using `async/await`? — robertklep, Mar 23 '17 at 07:13
@robertklep I have, in fact that was my first iteration before I started using async/await. It didn't work then either. As far as I've read, async/await is just syntactic sugar for doing exactly that. — neurozero, Mar 23 '17 at 07:30
[A work around was offered](https://github.com/facebook/jest/issues/1256) while async support was in preparation. Worth a try? — Roamer-1888, Mar 23 '17 at 07:47
@Roamer-1888 Nice thought, but I just tried it to no avail... — neurozero, Mar 23 '17 at 08:16
I think I figured it out, at least partially - My first function that I thought was working, wasn't. I just wasn't checking for errors. When I do check for errors, it fails every other time. Jest should have still caught that though because done() wouldn't have been called. — neurozero, Mar 23 '17 at 08:28
@jct from what I can see, `done` _would_ have been called, but the error wasn't being passed to it. It's strange though that Jest didn't catch the rejected promises. — robertklep, Mar 23 '17 at 08:50
@robertklep You are right, again - it would have been called regardless. I ended up finding a much better solution to this. I don't want to waste anybody else's time by reading through this monstrous unanswered question, but I'm pretty new to actively using Stack Overflow - should answer it and accept it or just delete this question? I doubt it adds much to the Stack Overflow community. Thanks for your help, I appreciate it. — neurozero, Mar 23 '17 at 09:02
@robertklep It might, I'll clean up the question too so it isn't so much to parse. — neurozero, Mar 23 '17 at 09:07
This answer for me worked. Every time. https://stackoverflow.com/a/11493324/728246 — zumzum, Feb 25 '20 at 00:33

neurozero · Answer 1 · 2017-03-23T09:38:29.693

9

I don't know why mongoose.connection.collections.<collection>.drop() randomly throws errors, but there is a simple way to remove all the documents in Mongoose, which works just fine for resetting the collection before tests:

beforeAll(async () => {
  await User.remove({})
})

Works every time.

edited Mar 23 '17 at 09:38

answered Mar 23 '17 at 09:13

neurozero

1,036
9
11

Woah, so simple! I'm sure this will help someone. – Roamer-1888 Mar 23 '17 at 12:52
I'm having the same problem when trying to clean up the database at the beginning, with every other run generating an 'database is in the process of being dropped.' in despite using promises. I tried using .drop() .remove({}) deleteMany({}).. always the same error – João Otero Sep 02 '18 at 23:42
I spent hours trying to solve this problem and finally an alternative. If someone knows why drop() method randomly throws erros, please post here. – Breno Teixeira Jun 15 '19 at 00:56
2022, Model.remove({}) is now deprecated, instead use Model.deleteMany({}); – Alvaro Rodriguez Scelza Aug 22 '22 at 20:15

João Otero · Answer 2 · 2018-09-03T06:05:00.637

I was having a similar issue while trying to drop the database in the beginning of the tests and populating the database right after. In the first run, collections would be created; in the next one, I would get an error of 'database is in the process of being dropped.'; ...and it was alternating like this.

I was also using an "in memory" Mongodb, running $ run-rs -v 4.0.0 -s (https://www.npmjs.com/package/run-rs) in a separate terminal window before running my Mocha tests. Also, I have Mongoose 5.2.0 and Mocha 5.1.1 here.

I've found out that Mongoose not necessarily executes the the drop commands immediately. Instead, it will schedule them for when the connection is up.

So, there can be a race condition in which the promise of the drop command is resolved and you move on in the code until reaching the instruction for creating your collections... but the drop command didn't finish running yet, and you'll get the error for the creation the new collections. The drop finish running and for the next time you run your test, your database (or collection) is already dropped, and hence you'll be able to insert the new collections again.

So, this is how I've solved...

Run in the before hook:

test.dropDb = function(done) {
    this.timeout(0)

    let database = your_MongoDB_URI
    mongoose.connect(database, function (err) {
        if (err) throw err;
        mongoose.connection.db.dropDatabase(function(err, result){
            console.log('database dropping was scheduled');
        });
    })
    mongoose.connection.on('connected', function() {
        setTimeout(function(){ done() }, 3000);
    });
}

Run in an nested before hook

test.createDb = function(done) {
    this.timeout(0)

    let createDb = require('./create-db.js')
    createDb() // here I call all my MyCollections.insertMany(myData)...
    .then( function() {
        done()
    })
    .catch( function(err) {
        assert( null == err)
    });
}

Run in the after hook

test.afterAll = function(done) {    
    mongoose.connection.close()
    .then( function() {
        done()
    })
}

I don't know why I had to explicitly close the connection in the after hook. I guess it has something to do with the way run-rs is programmed, but I didn't look into it. I just know that, in my case, closing the connection after the test was mandatory to get the expected behavior.

mongoose.connection.collections.collection.drop() throws error every other time

2 Answers2