12

I'm trying to learn the A Test Driven Approach with MongodB. The folder structure

enter image description here

A user.js to test in the src folder

const mongoose = require('mongoose');
mongoose.Promise = require('bluebird');
const Schema = mongoose.Schema;

const UserSchema = new Schema ({
    name: String
});

const User = mongoose.model('user', UserSchema);

module.exports = User;

Content of test_helper.js

const mongoose = require('mongoose');;

mongoose.connect('mongodb://localhost/users_test');

    mongoose.connection
    .once('open', () => {
        console.log('Connected to Mongo!');
        done()}) 
    .on('error', (error) => { 
        console.warn('Warning', error);
    });

create_test.js content

const assert = require('assert');
const User = require('../src/user');

describe('Creating records', () => {

    it('Saves a user', (done) => {
        const user = new User({ name: 'Ankur' });
        user.save()
                .then(() => {
                    assert(!user.isNew);
                    done();
                });

Now when i run npm test the test are getting passed.

Connected to Mongo!
  Creating records
    √ Saves a user (779ms)

But My doubt is how does Mocha knows to run the test_helper.js file first, Everytime. (Also naming this file to any other name doesn't change the behavior).

Also i'm not using any root-level hook.

i know mocha loads files recursively in each directory, starting with the root directory, and since everything here is one directory only so its not making any difference here.

Can someone please suggest or help, how does Mocha exactly know that test_helper.js (or any filename with the same content) should be running first.

Ankur Anand
  • 3,873
  • 2
  • 23
  • 44

3 Answers3

12

There is no default set order to how Mocha loads the test files.

When Mocha scans a directory to find files it it, it uses fs.readdirSync. This call is a wrapper around readdir(3), which itself does not guarantee order. Now, due to an implementation quirk the output of fs.readdir and fs.readdirSync is sorted on Linux (and probably POSIX systems in general) but not on Windows. Moreover, it is possible that the sorted behavior on Linux could eventually be removed because the documentation says fs.readdir is just readdir(3) and the latter does not guarantee order. There's a good argument to be made that the behavior observed on Linux is a bug (see the issue I linked to above).

Note that there is a --sort option that will sort files after Mocha finds them. But this is off by default.

The behavior you observe is explainable not merely by loading order but by execution order. Here is what happens:

  1. Mocha loads the test files and executes them. So anything that is at the top level of your file executes right away. This means that the code in test_helper.js executes right away. Every call to describe immediately executes its callback. However, calls to it record the test for later execution. Mocha is discovering your tests while doing this but not executing them right away.

  2. Once all files are executed, Mocha starts running the tests. By this time, the code in test_helper.js has already run and your test benefits from the connection it has created.

Major warning Connecting to a database is an asynchronous operation, and currently there is nothing that guarantees that the asynchronous operation in test_helper.js will have completed before the tests starts. That it works fine right now is just luck.

If this were me, I'd either put the connection creation in a global asynchronous before hook. (A global before hook appearing in any test file will be executed before any test whatsoever, even tests that appear in other files.) Or I'd use --delay and explicitly call run() to start the suite after the connection is guaranteed to be made.

Community
  • 1
  • 1
Louis
  • 146,715
  • 28
  • 274
  • 320
  • Thank you for such a detailed Answer. I've put it in before hook. it just that without before hook Also the test condition was showing this behavior.. was surprising for me – Ankur Anand Apr 05 '17 at 13:30
  • Ah, yes, if it is in a global `before` hook, you're avoiding the issue I described. It also conforms to the behavior I explained. The `before` hook necessarily has to execute before any test. It is not dependent on loading order. – Louis Apr 05 '17 at 13:33
  • fs.readdir and fs.readdirSync is sorted on Linux .. If i Could Vote twice would have done. for teaching me this .. because before you Answered Peter Müller (https://github.com/Munter) Mocha Contributor .. replied on this and i was still reading doc to see how it loads files in alphabetical order" and you answered the same – Ankur Anand Apr 05 '17 at 13:40
  • Peter Reply --- "Mocha loads and evaluates all files in the folder in alphabetical order. The files containing tests will queue up the test for mocha to run later. The content of test_helper.js contains no test and is executed immediately. This is why you get the DB connection before the tests start running It's incidental and you can't rely on the timing and the db connection being ready before you test If you want to guarantee a db connection before any tests start running you need to put the connection establishing code in a before hook and call back when it's up and running" – Ankur Anand Apr 05 '17 at 13:40
  • Yeah, I did a few tests, saw the sorted order and went "er... what?" because I knew it all ultimately depends on `readdir(3)` which does not guarantee order. Then I discovered the issue report, which explained the unexpected behavior. – Louis Apr 05 '17 at 13:49
1

It doesn't

Tests should not have a specific order. All test suites should be working as standalone agnostic to other suites. Inside the suite you can use "before" and "beforeEach" (or "after", "afterEach") in-order to create setup and teardown steps.

But if the order of the tests matters, something is broken in the design.

eavichay
  • 507
  • 2
  • 7
  • 1
    In the configuration file of mocha, you can specify which javascript files should be loaded before all tests begins, that's where you put your helper files. – eavichay Apr 05 '17 at 11:02
  • There is no other configuration done. Apart from installing and `scripts": { "test": "mocha" },` in package.json . create_test.js will needs mongodb connection to pass, this is not about test suites.standalone agnostic behavior – Ankur Anand Apr 05 '17 at 11:05
  • @AnkurAnand This answer is the correct one, mocha does _not_ know which file to load first, it's just coincidence. – Creynders Apr 05 '17 at 11:29
  • @Creynders .. create_test.js needs mongoose connection to validate .. if it is executed first then how validation happens without connection with the mongodb ? – Ankur Anand Apr 05 '17 at 12:02
  • @AnkurAnand I meant: you're just lucky. The files need to be loaded in _some_ order and apparently in your case it _does_ load them in the order that you need, but that's just a coincidence. Mocha does not _know_ it has to load them in that order. – Creynders Apr 05 '17 at 12:21
  • @Creynders Same behavior on 4 different system and also on cloud9 ide can't be a coincidence. :) .. – Ankur Anand Apr 05 '17 at 12:25
  • @Creynders Surprisingly for me also same behavior when i tried what AnkurAnand has mentioned. – user4890159 Apr 05 '17 at 12:30
  • @AnkurAnand yes it is. Maybe it's alphabetically loaded, or based on date created or... and then yes, it will be the same behaviour on all systems. My point is it's not something you can count on. Compare it to iterating over an object with for..in, 99% of the time it will look like it's always in the same order (i.e. the one in which you declared the keys) but then suddenly, it won't. This is the same thing. But seriously, if you have such a hard time believing me, no problem. Just remember this, when it suddenly doesn't work anymore. – Creynders Apr 05 '17 at 14:39
  • Here, have a look for yourself: https://github.com/mochajs/mocha/blob/master/lib/utils.js#L657 – Creynders Apr 05 '17 at 14:47
  • @Creynders It's wasn't thing for believe or not.Question was why such exception is been observed. see the Above answer by Louis and check comment's you will understand why this answer was not the thing that was being looked through this question .. Loading Order where it doesn't matter and where you can see .. Peter Müller is one of the core contributor of Mocha and i've updated in comment section.. His comment on this issue, too. and why it was happening . This Answer just provided the behavior of mocha library as general which was not the point. Anyway Louis detailed it. Thanks for Help :) – Ankur Anand Apr 05 '17 at 14:54
  • While your comment is absolutely correct, with regards to that all tests should be runnable individually, when you are running a suite of tests, you will frequently want to run them in a specific order. Tests are usually performed from lowest level (test the lowest level functions in the system), then you test the higher level functions that depend on that lower level functionality. – Eric Blade Jan 29 '18 at 01:16
0

There is a very easy way to load tests sequentially.

Step 1 : Set up a test script in package.json: e.g.

"scripts": {
    "test": "mocha ./tests.js"
  }

Let us assume that tests.js is a file which defines the order to execute tests.

require("./test/general/test_login.js");
require("./test/Company/addCompany.js");
...
...

So here test_login will run first and then others one by one.

Step 2: Then run tests with:

$ npm test