Introduction
I am getting stuck with database Promises I am using inside the Jest testing framework. Things are running in the wrong order, and following some of my latest changes, Jest is not finishing correctly because an unknown asynchronous operation is not being handled. I am fairly new to Node/Jest.
Here is what I am trying to do. I am setting up Jest inside a multiple Docker container environment to call internal APIs in order to test their JSON outputs, and to run service functions in order to see what change they make against a test-environment MySQL database. To do that, I am:
- using the
setupFilesAfterEnvJest configuration option to point to a setup file, which I believe should be run first - using the setup file to destroy the test database (if it exists), to recreate it, and then to create some tables
- using
mysql2/promiseto carry out operations on the database - using a
beforeEach(() => {})in a test to truncate all tables, in readiness for inserting per-test data, so that tests are not dependent on each other
I can confirm that the setup file for Jest is being run before the first (and only) test file, but what is odd is that a Promise catch() in the test file appears to be thrown before a finally in the setup file.
I will put my code down first, and then speculate on what I vaguely suspect to be a problem.
Code
Here is the setup file, nice and straightforward:
// Little fix for Jest, see https://stackoverflow.com/a/54175600
require('mysql2/node_modules/iconv-lite').encodingExists('foo');
// Let's create a database/tables here
const mysql = require('mysql2/promise');
import TestDatabase from './TestDatabase';
var config = require('../config/config.json');
console.log('Here is the bootstrap');
const initDatabase = () => {
let database = new TestDatabase(mysql, config);
database.connect('test').then(() => {
return database.dropDatabase('contributor_test');
}).then(() => {
return database.createDatabase('contributor_test');
}).then(() => {
return database.useDatabase('contributor_test');
}).then(() => {
return database.createTables();
}).then(() => {
return database.close();
}).finally(() => {
console.log('Finished once-off db setup');
});
};
initDatabase();
The config.json is just usernames/passwords and not worth showing here.
As you can see this code uses a utility database class, which is this:
export default class TestDatabase {
constructor(mysql, config) {
this.mysql = mysql;
this.config = config;
}
async connect(environmentName) {
if (!environmentName) {
throw 'Please supply an environment name to connect'
}
if (!this.config[environmentName]) {
throw 'Cannot find db environment data'
}
const config = this.config[environmentName];
this.connection = await this.mysql.createConnection({
host: config.host, user: config.username,
password: config.password,
database: 'contributor'
});
}
getConnection() {
if (!this.connection) {
throw 'Database not connected';
}
return this.connection;
}
dropDatabase(database) {
return this.getConnection().query(
`DROP DATABASE IF EXISTS ${database}`
);
}
createDatabase(database) {
this.getConnection().query(
`CREATE DATABASE IF NOT EXISTS ${database}`
);
}
useDatabase(database) {
return this.getConnection().query(
`USE ${database}`
);
}
getTables() {
return ['contribution', 'donation', 'expenditure',
'tag', 'expenditure_tag'];
}
/**
* This will be replaced with the migration system
*/
createTables() {
return Promise.all(
this.getTables().map(table => this.createTable(table))
);
}
/**
* This will be replaced with the migration system
*/
createTable(table) {
return this.getConnection().query(
`CREATE TABLE IF NOT EXISTS ${table} (id INTEGER)`
);
}
truncateTables() {
return Promise.all(
this.getTables().map(table => this.truncateTable(table))
);
}
truncateTable(table) {
return this.getConnection().query(
`TRUNCATE TABLE ${table}`
);
}
close() {
this.getConnection().close();
}
}
Finally, here is the actual test:
const mysql = require('mysql2/promise');
import TestDatabase from '../TestDatabase';
var config = require('../../config/config.json');
let database = new TestDatabase(mysql, config);
console.log('Here is the test class');
describe('Database tests', () => {
beforeEach(() => {
database.connect('test').then(() => {
return database.useDatabase('contributor_test');
}).then (() => {
return database.truncateTables();
}).catch(() => {
console.log('Failed to clear down database');
});
});
afterAll(async () => {
await database.getConnection().close();
});
test('Describe this demo test', () => {
expect(true).toEqual(true);
});
});
Output
As you can see I have some console logs, and this is their unexpected order:
- "Here is the bootstrap"
- "Here is the test class"
- Tests finish here
- "Failed to clear down database"
- "Finished once-off db setup"
- Jest reports "Jest did not exit one second after the test run has completed. This usually means that there are asynchronous operations that weren't stopped in your tests."
- Jest hangs, requires ^C to exit
I want:
- "Here is the bootstrap"
- "Finished once-off db setup"
- "Here is the test class"
- No error when calling
truncateTables
I suspect that the database error is that the TRUNCATE operations are failing because the tables do not exist yet. Of course, if the commands ran in the right order, they would!
Notes
I originally was importing mysql instead of mysql/promise, and found from elsewhere on Stack Overflow that without promises, one needs to add callbacks to each command. That would make the setup file messy - each of the operations connect, drop db, create db, use db, create tables, close would need to appear in a deeply nested callback structure. I could probably do it, but it is a bit icky.
I also tried writing the setup file using await against all the promise-returning db operations. However, that meant I had to declare initDatabase as async, which I think means I can no longer guarantee the whole of the setup file is run first, which is in essence the same problem as I have now.
I have noticed that most of the utility methods in TestDatabase return a promise, and I am pretty happy with those. However connect is an oddity - I want this to store the connection, so was confused about whether I could return a Promise, given that a Promise is not a connection. I have just tried using .then() to store the connection, like so:
return this.mysql.createConnection({
host: config.host, user: config.username,
password: config.password
}).then((connection) => {
this.connection = connection;
});
I wondered if that should work, since the thenable chain should wait for the connection promise to resolve before moving onto the next thing in the list. However, the same error is produced.
I briefly thought that using two connections might be a problem, in case tables created in one connection cannot be seen until that connection is closed. Building on that idea, maybe I should try connecting in the setup file and re-using that connection in some manner (e.g. by using mysql2 connection pooling). However my senses tell me that this is really a Promise issue, and I need to work out how to finish my db init in the setup file before Jest tries to move on to test execution.
What can I try next? I am amenable to dropping mysql2/promise and falling back to mysql if that is a better approach, but I'd rather persist with (and fully grok) promises if at all possible.
