@swc/jest and typescript does not allow jest to run tests

Question

I'm using react, typescript and SWC to compile my project, also using @swc/jest as transformer in my jest config like below:

module.exports = {
  ...some config

  transform: {
    '^.+\\.(t|j)sx?$': '@swc/jest',
  },
}

here is my typescript config:

{
  "compilerOptions": {
    "outDir": "lib",
    "module": "ESNext",
    "target": "ES2022",
    "sourceMap": true,
    "jsx": "react",
    "declaration": true,
    "isolatedModules": true,
    "strict": true,
    
    "moduleResolution": "node",
    "allowSyntheticDefaultImports": true,
    "esModuleInterop": true,
    "resolveJsonModule": true,
    "forceConsistentCasingInFileNames": true,
    "skipLibCheck": true
  },
  "exclude": ["node_modules", "lib", "dist"]
}

As you can see I choose ESNext as module and my target is ES2022. But when I try to run my test command, I get this error of: Error [ERR_REQUIRE_ESM]: require() of ES Module .... module not supported

$ jest
Error [ERR_REQUIRE_ESM]: require() of ES Module <MY_PROJECT>/node_modules/wrap-ansi/index.js from <MY_PROPJECT>/node_modules/cliui/build/index.cjs not supported.
Instead change the require of index.js in <MY_PROJECT>/node_modules/cliui/build/index.cjs to a dynamic import() which is available in all CommonJS modules.
    at Object.<anonymous> some long long stack trace...
error Command failed with exit code 1.

I have followed this: Jest module not defined and this https://github.com/jestjs/jest/issues/11453 and also this Error [ERR_REQUIRE_ESM]: require() of ES Module not supported which is using ts-jest. I also installed ts jest just for trying out if it would work but no luck so I removed it.

As I mentioned I rely on @swc/jest for transforming my modules but I really don't know why it is not working.

I also tried ignoring from those 2 packages that causes this issue in my jest config file like below:

  transformIgnorePatterns: ['<rootDir>/node_modules/(?!(cliui|wrap-ansi)/*)'],

but still no luck!

The very interesting thing is, if I remove yarn.lock and my node_modules then run my test command -> it works, but as soon as I install/remove additional packages (which adjusts my yarn.lock file) it gives me the same error. also this happens in my GitHub actions workflow as well.

What am I missing here?