How can I use yarn workspaces to run tests for all workspaces, even if one fails?

Question

I am using yarn workspaces (yarn version 1.22.19) and I would like to run tests for all workspaces, without stopping even if tests fail for one of the workspaces. This is so I can collect all failing tests across all workspaces in one run. I'm running the tests on a github action.

I am running the following command: yarn workspaces run test --passWithNoTests All workspaces have a test script in the package.json that runs the tests with Jest. Jest returns an exit code of 1 when tests fail. This causes the yarn workspaces run command to fail and stop. I would like it to continue and fail only after running tests for all workspaces.

How can I make the yarn workspaces run continue even if tests fail for one of the workspaces, yet still have it fail at the end?

Edit:

I am running bash. Using workarounds like set -e or || true might help swallow the error, but I do want the command to fail ultimately, I just want it to fail after running all tests.

For example:

Say I have 3 workspaces - workspace a, workspace b and workspace c. All of them have the following script in their package.json:

test: "jest"

Say tests pass for workspace a and workspace c, but fail for workspace b. My desired result is that running yarn workspaces run test will run tests for all workspaces (and not stop after tests fail for workspace b) but for it to fail after running all tests.

Here is my github workflow. It just installs dependencies and runs the test script which runs the command yarn workspaces run test --passWithNoTests.

name: Run All Tests

on:
    pull_request:
        branches: ['develop']

jobs:
    build:
        runs-on: ubuntu-latest

        strategy:
            matrix:
                node-version: [16.x]
                # See supported Node.js release schedule at https://nodejs.org/en/about/releases/

        steps:
            - uses: actions/checkout@v3
            - name: Use Node.js ${{ matrix.node-version }}
              uses: actions/setup-node@v3
              with:
                  node-version: ${{ matrix.node-version }}
                  cache: 'npm'
            - name: Install dependencies
              run: yarn install --frozen-lockfile
            - name: Run tests
              run: yarn test

Please add your workflow to your question to highlight the runners and shells that you're using. If you're using `ubuntu-*` runners with the default shell i.e. bash, you can do `command || true` to return true even if `command` fails. Alternatively, if you're running multiple commands then you can disable exiting on failure with `set +e` before running those commands and then enable it i.e. `set -e`. — Azeem, Feb 03 '23 at 06:05
Does this answer your question? [Bash ignoring error for a particular command](https://stackoverflow.com/questions/11231937/bash-ignoring-error-for-a-particular-command) — Azeem, Feb 03 '23 at 06:27
@Azeem Added some clarification. Using `set -e` or `|| true` won't fully solve what I'm trying to accomplish. — Oded, Feb 03 '23 at 14:54
Right. That's doable by storing Boolean output (true/false) for each step using `GITHUB_OUTPUT` and then ANDing all the Boolean results in the last step. It would really help if you added n skeleton workflow representing your exact use case. — Azeem, Feb 03 '23 at 15:35
@Azeem I added the workflow but there's not a lot to it. It's not possible to save the output of each workflow's `yarn test` results, since for github it's just one step, it's not a step per workspace. — Oded, Feb 03 '23 at 22:49
Oh, right. Then, it's one level above and should be configured as part of your tests. This might be helpful: https://stackoverflow.com/a/62973004/7670262. I'm not well-versed enough in this to comment and provide any guidance. You might want to include your `package.json` in your question as well. Hopefully, someone knowledgeable enough would guide you accordingly. Good luck! — Azeem, Feb 04 '23 at 07:21

score 1 · Accepted Answer · answered Feb 08 '23 at 20:27

For future reference, this is what I ended up doing:

Add a test:ci script to all package.json file with the following defintion:

"test:ci": "jest --ci --reporters=jest-junit --reporters=default --passWithNoTests || true"

This makes it so the command passes even if tests fails.

Use a jest-junit test reporter to output an xml with test results.
In the action, run yarn workspaces run test which runs tests for all workspaces (packages).
Use the dorny/test-reporter@v1 to collect all test result xml files into a nice view.
Set fail-on-error: 'true' for the action dorny/test-reporter@v1 which will make the step fail if any test failed.

Here's the full github workflow:

# This workflow will do a clean installation of node dependencies, cache/restore them, build the source code and run tests across different versions of node
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-nodejs

name: Build & Test

on:
    pull_request:
        branches: ['develop']

# cancel any previous runs that are still in progress if a new commit is pushed
concurrency:
    group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
    cancel-in-progress: true

jobs:
    test:
        runs-on: ubuntu-latest
        strategy:
            matrix:
                node-version: [16.x]
        steps:
            - uses: actions/checkout@v3
            - name: Use Node.js ${{ matrix.node-version }}
              uses: actions/setup-node@v3
              with:
                  node-version: ${{ matrix.node-version }}
                  cache: 'npm'
            - name: Install dependencies
              run: yarn install --frozen-lockfile
            - name: Run tests
              run: yarn test:ci
            - name: Unlink all symbolic links # so we don't go over the same file twice
              if: success() || failure() # run this step even if previous step failed
              run: find node_modules -type l -exec unlink {} \;
            - name: Test Report
              uses: dorny/test-reporter@v1
              if: success() || failure() # run this step even if previous step failed
              with:
                  name: Jest Test Results # Name of the check run which will be created
                  path: '**/jest-junit.xml'
                  reporter: jest-junit # Format of test results
                  list-suites: 'failed'
                  list-tests: 'failed'
                  fail-on-error: 'true'

This achieves everything I was looking for - running all tests for all workspaces, even if some fail, while still failing the workflow.

How can I use yarn workspaces to run tests for all workspaces, even if one fails?

1 Answers1