91

How would I mock out the database in my node.js application, which in this case uses mongodb as the backend for a blog REST API ?

Sure, I could set the database to a specific testing -database, but I would still save data and not test my code only, but also the database, so I am actually not doing unit testing but integration testing.
So what should one do? Create database wrappers as a middle layer between application and db and replace the DAL when in testing?

// app.js  
var express = require('express');
    app = express(),
    mongo = require('mongoskin'),
    db = mongo.db('localhost:27017/test?auto_reconnect');

app.get('/posts/:slug', function(req, res){
    db.collection('posts').findOne({slug: req.params.slug}, function (err, post) {
        res.send(JSON.stringify(post), 200);
    });
});

app.listen(3000);

// test.js
r = require('requestah')(3000);
describe("Does some testing", function() {

  it("Fetches a blogpost by slug", function(done) {
    r.get("/posts/aslug", function(res) {
      expect(res.statusCode).to.equal(200);
      expect(JSON.parse(res.body)["title"]).to.not.equal(null);
      return done();
    });

  });
));
Community
  • 1
  • 1
Industrial
  • 41,400
  • 69
  • 194
  • 289

6 Answers6

146

I don't think database related code can be properly tested without testing it with the database software. That's because the code you're testing is not just javascript but also the database query string. Even though in your case the queries look simple you can't rely on it being that way forever.

So any database emulation layer will necessarily implement the entire database (minus disk storage perhaps). By then you end up doing integration testing with the database emulator even though you call it unit testing. Another downside is that the database emulator may end up having a different set of bugs compared to the database and you may end up having to code for both the database emulator and the database (kind of like the situation with IE vs Firefox vs Chrome etc.).

Therefore, in my opinion, the only way to correctly test your code is to interface it with the real database.

slebetman
  • 109,858
  • 19
  • 140
  • 171
  • 1
    You know, you make a good point. While unit testing serves a phenomenal purpose (i.e. isolation), you've made a strong point for integration testing. – Mike Perrenoud Oct 07 '15 at 00:14
  • 5
    @MichaelPerrenoud: I like the rule set forth by christkv's answer: **"Don't mock anything you don't own"**. While it doesn't go into detail why it's a bad idea, it's an easy rule to remember. – slebetman Oct 07 '15 at 02:35
  • 2
    I disagree with this answer, in meteorjs they set up a testing DB somehow when running tests (I assume it's not a mock library, but a temporary file) and it's very convenient. It would be very useful to have an object that behaves exactly like a mongodb and cleans after itself. Whether it's all in memory or a temporary file is implementation detail, so you don't have to duplicate code. I do agree that the people who make the driver should be the ones making the mock object. – Uri Sep 04 '16 at 07:51
  • 1
    I think I also disagree with this answer. OP's question was simplistic in that it's only a controller and the database. What about all the application logic that makes servers servers and not just pipes from the DB to the internet? Imagining you have an object that you want to validate and then put into the database, isn't it fair to imagine something like an integration test for checking that the interaction still works and unit tests to check all the validation? Taking this answer literally, how can one test anything with data that originates in the database? How long do your builds take?? – user2152081 Feb 16 '17 at 00:23
  • You forgot `PhantomJS` in your list of browsers, which is a fake-browser, purely written in JavaScript. It's only purpose is automated testing. And yes, it has bugs that you won't find in a normal browser ... – SimonSimCity May 22 '17 at 06:40
  • 2
    Look, mocking DB calls is profoundly silly and only leads to busy work. I think people are missing the whole point of tests. It's to show that input matches output. For a function calling a DB, You can't know that for sure unless that test involves the db call. The best way to do that is to have your test config point to an instance of MongoDB that is running on the memory. That way you don't have to deal with disk I/O and you also fulfill the major point of 'mocking', which is to run from memory. – Glstunna Aug 23 '17 at 11:05
  • Is there a test framework for JS that provides e.g. a Rails-like experience of test db config & setup? – Dogweather Sep 25 '20 at 19:15
  • I don't agree, but I also think that this isn't an answer to the question. It's an opinion about whether the question is valid in the first place. That being the case, I don't think it deserves to be the accepted answer. – Adam Apr 03 '23 at 22:18
50

There is a general rule of thumb when it comes to mocking which is

Don't mock anything you don't own.

If you want to mock out the database hide it behind an abstracted service layer and mock that layer. Then make sure you integration test the actual service layer.

Personally I've gone away from using mocks for testing and use them for top-to-bottom design helping me drive development from the top towards the bottom mocking out service layers as I go and then eventually implementing those layers and writing integration tests. Used as a test tool they tend to make your test very brittle and in the worst case leads to a divergence between actual behavior and mocked behavior.

Vanilla
  • 51
  • 1
  • 3
  • 9
christkv
  • 4,370
  • 23
  • 21
  • Sure you can hide it behind a repository or gateway and use the mock to drive your test driven approach and to isolate your unit tests...what do you mean you don't use mocks for testing then? You're gonna keep that mocked gateway/repository still in your tests then somehow use an interface to specify the real implementation in you repository through an interface right? – PositiveGuy Sep 06 '15 at 23:50
  • How do you handle third party APIs? A VCR-like helper? – Dogweather Sep 25 '20 at 19:16
  • You're the best Sir! – Marco Lazzeri Oct 13 '20 at 17:30
46

I don't agree with the selected answer or other replies so far.

Wouldn't it be awesome if you could catch errors spawned by the chaotic and many times messy changes made to DB schemas and your code BEFORE it gets to QA? I bet the majority would shout heck yes!

You most certainly can and should isolate and test you DB schemas. And you don't do it based on an emulator or heavy image or recreation of you DB and machine. This is what stuff like SQLite is for just as one example. You mock it based on an in memory lightweight instance running and with static data that does not change in that in memory instance which means you are truly testing your DB in isolation and you can trust your tests as well. And obviously it's fast because it's in memory, a skeleton, and is scrapped at the end of a test run.

So yes you should and you should test the SCHEMA that is exported into a very lightweight in memory instance of whatever DB engine/runtime you are using, and that along with adding a very small amount of static data becomes your isolated mocked DB.

You export your real schemas from your real DB periodically (in an automated fashion) and import/update those into your light in memory DB instance before every push to QA and you will know instantly if any latest DB changes done by your DB admins or other developers who have changed the schema lately have broken any tests .

As for the person who replied with the "don't mock anything you don't own". I think he meant to say "don't test anything you don't own". But you DO mock things you do not own! Because those are the things not under test that need to be isolated!

This is what many test driven teams do all the time. You just have to understand the how.

General Grievance
  • 4,555
  • 31
  • 31
  • 45
PositiveGuy
  • 17,621
  • 26
  • 79
  • 138
  • 7
    I would love to upvote, but I can't if it doesn't mitigate the question and provide a solution. Please update your post when you get a chance. – Shanimal Mar 30 '16 at 16:50
  • 11
    2018 and still there isn't a "how", was certainly interested to read more. – MacK Apr 27 '18 at 12:17
  • 4
    I have down voted since 3 years onward there is still no how. – ian Jun 28 '18 at 05:43
  • 1
    6 years later and still nothing about the HOW... Why making us waiting for so long ? I down voted this answer, and upvoted the [next one](https://stackoverflow.com/a/12580235/6515775) (which is now above :) just because of this. I'll reconsider this when we'll get the HOW – mr.mams Nov 25 '21 at 21:44
5

My preferred approach to unit test DB code in any language is to access Mongo through a Repository abstraction (there's an example here http://iainjmitchell.com/blog/?p=884). Implementations will vary in terms of DB specific functionality exposed but by removing all the Mongo code from your own logic you're in a position to Unit Test. Simply replace the Mongo Repository implementation with a stubbed out version which is trivially easy. For instance, just store objects in a simple in-memory dictionary collection.

You'll get the benefits of unit testing your own code this way without DB dependencies but you'll still need to do integration tests against the main DB because you'll probably never be able to emulate the idiosyncrasies of the real database as others have said here. The kind of things I've found are as simple as indexing in safe mode vs without safe mode. Specifically, if you have a unique index your dummy memory implementation might honour that in all cases, but Mongo won't without safe-mode.

So whilst you'll still need to test against the DB for some operations, you'll certainly be able to unit test your own logic properly with a stubbed out Repository implementation.

cirrus
  • 5,624
  • 8
  • 44
  • 62
  • but at some point your real implementation code has to reference the real data calls. I assume you're injecting the Interface into your repository to the real Data Layer query code? – PositiveGuy Sep 06 '15 at 23:51
  • 1
    Try using Docker. I have solved years of nightmarish configuration issues with databases and package installs by using Docker to run containers running databases initialized with specific test data for the scenario. In fact I run stacks of 3 containers: one with the DB, one with the application code and one with the test driver. If your data set is moderate in size, you can even spin up parallel instances of these stacks, considerably shortening your testing cycle. – ovo Oct 23 '15 at 18:45
4

The purpose of mocking is to skip the complexity and unit test own code. If you want to write e2e tests then use the db.

Writing code to setup/teardown a testing DB for unit testing is technical debt and incredibly unsatisfying.

There are mock libraries in npm:

mongo - https://www.npmjs.com/package/mongomock

mongoose - https://www.npmjs.com/package/mockgoose

If those don't support the features you need, then yes you may need to use the real thing.

Script_Coded
  • 709
  • 8
  • 21
Michael Cole
  • 15,473
  • 7
  • 79
  • 96
  • 1
    mongo-mock does not support aggregation operations for example which can be a problem i started using it and at the end i ended up using the real software with a testOnly database – zardilior Jul 07 '19 at 01:54
  • Also when changing fro mongo-mock to mongodb a lot of stuff broke down so it defeated the purpose – zardilior Jul 07 '19 at 01:56
3

I had this dilemma and chosen to work with a test DB and clean it every time the test begins. (how to drop everything: https://stackoverflow.com/a/25639377/378594)

With NPM you can even make a test script that creates the db file and cleans it up after.

Uri
  • 25,622
  • 10
  • 45
  • 72
  • this is actually a nice approach and I would recommand anyone doing serious CI/CD pipelining to do so. With today's JavaScript tooling we're able to create/drop test-databases easily before and after running tests. I would argue whether this approach is suitable while developing (you don't want to drop & create your database on every code change), but there's other solution for that. At least it solves the problem of having to maintain seperate plugin integrations that mock data and glueing them to your test environment. – Nicky Oct 26 '17 at 14:32