eslint doesn't recognize rule violations in test files and built-in testrunner only picks up test files from the first depth

Question

I'm using

Node 18.4
npm 8.13.1
Pop!_OS 22.04 LTS

and have a sample project where I would like to try out the new built-in testrunner. Given the following structure

.
├── lib
│   └── index.ts
├── test
│   └── aFolder
│       ├── aSubFolder
│       │   └── ignoredTests.ts
│       └── pickedUpTests.ts
├── .eslintrc.json
├── .gitignore
├── package.json
└── tsconfig.json

Now I will describe the file content but I also created a reproduction repository that you can find here

Content of index.ts

export {};

Content of .eslintrc.json

{
  "extends": "es/browser"
}

Content of package.json

{
  "name": "wildcard-reproduction",
  "version": "0.0.1",
  "scripts": {
    "lint": "npx eslint ./**/*.ts",
    "test": "node --test --require ts-node/register ./test/**/*Tests.ts"
  },
  "dependencies": {
    "@types/node": "18.0.0"
  },
  "devDependencies": {
    "eslint": "7.32.0",
    "eslint-config-es": "4.2.0",
    "ts-node": "10.8.1",
    "typescript": "4.7.4"
  }
}

Content of tsconfig.json

{
  "compilerOptions": {
    "baseUrl": ".",
    "declaration": true,
    "esModuleInterop": true,
    "lib": [ "es2020", "dom" ],
    "module": "commonjs",
    "outDir": "build",
    "resolveJsonModule": true,
    "strict": true,
    "target": "es2020"
  },
  "include": [
    "./**/*.ts"
  ],
  "exclude": [
    "./build"
  ]
}

Content of pickedUpTests.ts

import assert from 'assert/strict';
import test from 'node:test';

test('picked up by testrunner', async t => {
    await t.test('passes.', async () => {
        assert.ok(true);
    });
});

Content of ignoredTests.ts

import assert from 'assert/strict';
import test from 'node:test';

test('ignored by testrunner', async t => {
    await t.test('fails.', async () => {
        assert.ok(false);
    });
});

The code contains multiple linter errors so eslint shoud fail. But when running npm run lint everything seems to be fine

There are multiple tests in multiple folders inside the test folder. When running npm run test the testrunner only picks up the tests in the first depth

Something seems to be broken but I can't figure out what exactly. I think the npm scripts are fine. I'm expecting failing tests and linter errors but both are passing...

The funny thing: When deleting the lib folder, at least the linter seems to work for the test files ...

Does someone know what's wrong here?

I would suggest asking one thing at a time, ESLint and Node's new built-in test runner module are completely separate tools. — jonrsharpe, Jun 29 '22 at 13:37
@jonrsharpe yes, you are right. But I think the problem is the same... — , Jun 29 '22 at 13:37
For ESLint you can just do `"lint": "eslint ."` (you don't need `npx` in your package scripts because `./node_modules/.bin/` is already added to the path). For the tests you probably need to enable the `globstar` option, see e.g. https://stackoverflow.com/a/28199633/3001761. — jonrsharpe, Jun 29 '22 at 13:49

score 1 · Accepted Answer · answered Jul 02 '22 at 03:21

In the future please only include a single question at a time.

ESLint false positives

You are using an incompatible version of eslint with typescript. Furthermore your eslint rules don't specify a rule for the example (export {}) shown above.

WARNING: You are currently running a version of TypeScript which is not officially supported by @typescript-eslint/typescript-estree.

You may find that it works just fine, or you may not.

SUPPORTED TYPESCRIPT VERSIONS: >=3.3.1 <4.6.0

YOUR TYPESCRIPT VERSION: 4.7.4

Please only submit bug reports when using the officially supported version.

This is caused in part by an outdated plugin included in your project "eslint-config-es", which in turn installs outdated versions of eslint and more. The standard way to incorporate eslint into your typescript project would be to allow npm to find the correct packages for the version of node you have installed. You can achieve this by doing the following:

npm install --save-dev eslint typescript @typescript-eslint/parser @typescript-eslint/eslint-plugin

Now a warning should not appear.

Next, is to set up a smoke test. All I saw in your example was a file exporting an empty object, so I updated index.ts to include a smoke test—var x = true and ran the linter. This was the result:

/wildcard-reproduction/lib/index.ts
  1:1  error    Unexpected var, use let or const instead  no-var
  1:5  warning  'x' is assigned a value but never used    @typescript-eslint/no-unused-vars

✖ 2 problems (1 error, 1 warning)
  1 error and 0 warnings potentially fixable with the `--fix` option.

The linter seems to be working correctly. Any additional rules you would like to include should be added to your .eslintrc.json.

For reference here is the eslint I added based on eslint documentation:

{
  "root": true,
  "parser": "@typescript-eslint/parser",
  "plugins": [
    "@typescript-eslint"
  ],
  "extends": [
    "eslint:recommended",
    "plugin:@typescript-eslint/recommended"
  ]
}

Here is the package.json devDependencies updated when running the npm install command from earlier.

"devDependencies": {
    "@typescript-eslint/eslint-plugin": "^5.30.3",
    "@typescript-eslint/parser": "^5.30.3",
    "eslint": "^8.19.0",
    "ts-node": "10.8.1",
    "typescript": "^4.7.4"
  }

node:test recursive

You are using an experimental feature, therefore there are expected to be bugs. One of these bugs is glob parsing during test file discovery. This should work, but doesn't:

    "test": "node --test --require ts-node/register test/**/*.ts"

A temporary fix for the example you have is the following:

    "test": "node --test --require ts-node/register test/**/**/*.ts"

References

https://typescript-eslint.io/docs/linting/

https://nodejs.org/api/test.html#test-runner-execution-model

for the testrunner: I will search for an issue mentioning broken glob parsing — Question3r, Jul 03 '22 at 11:47
for the linter: Do you what to do for the `eslint-config-es` package? I cloned it and ran `npx npm-check -u` but there wasn't anything special to update. Do you know the outdated parts? — Question3r, Jul 03 '22 at 11:50
For the testrunner: https://github.com/nodejs/node/issues/43675 — , Jul 04 '22 at 11:37

eslint doesn't recognize rule violations in test files and built-in testrunner only picks up test files from the first depth

1 Answers1

ESLint false positives

node:test recursive

References