9

Question:

How to import JavaScript files in NodeJS using file paths from project root? (instead of relative file paths)

In other words, my team is trying to transition from Syntax Style A to Syntax Style B (shown below):

Syntax Style A

const source = require('../../../../../../source.js');

Syntax Style B

const source = require('src/source.js'); // "src" is a direct subfolder of the project root

I have reviewed a lot of posts on this topic, and none of them can satisfy my project's requirements...


Project Requirements:

  • Requirement #1: No external packages (such as Babel.js or Webpack), since disk space is limited. Also, we cannot install anything with "npm install".
  • Requirement #2: A JavaScript file should be able to import another JavaScript file using "project root" file paths, not relative file paths. (This will avoid long relative paths such as "../../../../../../../script.js", which can become unmanageable.)
  • Requirement #3: Right-clicking the imported class anywhere in the source code should bring up a context menu, with an option to "Go to Declaration" (which takes us to the imported file directly). Also, moving a JavaScript file to a different folder in the IDE should automatically update all references. We can use any IDE.
  • Requirement #4: To have a consistent way of importing scripts, we will be using NodeJS's "require" keyword to import (instead of JavaScript's "import" keyword).

Setup:

Below is my setup. Package.json was generated by WebStorm, and I haven't changed it.

Screenshot_of_Project_Structure

Project_Root
├─-─ src
│    └── a1
│       └── a2
│           └── a3
│               └── Worker.js
├──- subFolder1
│    └── SubWorker.js
├─ main.js
└─ package.json

main.js

const Worker = require('./src/a1/a2/a3/Worker.js');
Worker.doWork();

Worker.js

// If I replace the following with Project Root path, such as:
//    const SubWorker = require('src/subFolder1/SubWorker.js');
// The script will fail with "Error: Cannot find module './src/subFolder1/SubWorker.js'."

const SubWorker = require('../../../subFolder1/SubWorker.js');

module.exports =
class Worker { 

        static doWork() {
                console.log("doing work");
                SubWorker.doMoreWork();
        }
}

SubWorker.js

module.exports =
class SubWorker {

        static doMoreWork() {
                console.log("doing more work");
        }

};

Attempt #1: path.resolve

I tried the "path.resolve" solution, as suggested by this post:


Attempt #2: app-module-path

  • Importing using relative paths in Node.js with ES Modules
    • Using app-module-path solves all three requirements, but the program will not run (which is an implied requirement): ⛔Error: Cannot find module 'src/subFolder1/SubWorker.js'
    • Error was reproducible in both WebStorm 2022.3.2 and VS Code 1.76.2

Attempt #3: dynamic import()

Worker.js

const rootPath = 'src/subFolder1';
const SubWorker = require(`${rootPath}/SubWorker.js`);


Attempt #4: subpath imports


Expected Outcome:

Since NodeJs is one of the most important backend languages, I was expecting excellent IDE support for imports... Referencing a source file from Project Root should be a simple operation that is handled by the IDE (I shouldn't need to use Babel.js or Webpack). Am I doing something wrong here?

I don't know what I am doing wrong, since I have already followed standard practices:

  • I used latest versions of the most popular NodeJS IDEs (WebStorm 2022.3.2 and VS Code 1.76.2)
  • I used WebStorm's "Create a new NodeJS project" wizard to create the project
  • I used the latest NodeJS (v18.15.0). (on Windows 10, v21H2)

Thanks!


Related Posts:

The Stack Overflow posts below are relevant, but do not address the specific issue in this post. (Please kindly search by the title on Stack Overflow, since the spam filter limited the number of links in this post, thanks!)

  • Title: NodeJS - Relative File Paths
    • Failed the same way as my Attempt #1 above
  • Title: Project root based relative paths for NodeJS modules
    • Violated Requirement #4 (the desired solution should use "require" instead of "import")
  • Title: is it possible to get rid of the relative paths on imports?
    • The top answer uses Babel.js, which violates Requirement #1
  • Title: How to import a JavaScript file to another JavaScript file
    • Does not address the question of relative path vs "project root" path
  • Title: How to use absolute/dynamic paths in NodeJs
    • Violated Requirement #4 (the desired solution should use "require" instead of "import")
  • Title: how to use absolute paths instead of relative paths in React or Next.js?
    • We are not using React or Next.js here.
  • Title: JavaScript Import file by absolute path
    • Violated Requirement #4 (the desired solution should use "require" instead of "import")
  • Title: javascript file with absolute file path
    • The script here was launched from the client-side web browser. This post deals with nodeJS scripts that are launched from the server-side backend.

Heretic Monkey
  • 11,687
  • 7
  • 53
  • 122
howard_9
  • 413
  • 3
  • 15
  • Did you look at https://smellycode.com/require-with-aliases/ ? – Anatoly Mar 21 '23 at 21:00
  • 1
    Hi @Anatoly, "require-with-aliases" involves the installation of an external package called "module-alias", which violates Project Requirement #1 (no external packages). My project manager believes that importing a file should be a well-supported feature in WebStorm, since NodeJS is the leading backend language with 48% market share. Therefore, just like other leading langauges such as Java/Python, WebStorm or VS Code should have a way to allow out-of-the-box "project root" imports (especially when using the IDE "New NodeJs Project" wizard). – howard_9 Mar 21 '23 at 21:19
  • 2
    You seem to think this should be a feature of VS Code or webStorm. That is not simply the case. Since you've specified no outside packages can be installed, what you're looking for is something that nodejs supports all by itself. And, such a feature that you are asking for does not exist in nodejs. – jfriend00 Mar 21 '23 at 21:23
  • FYI, none of the "Related Posts" in your question seem to have links associated with them. – jfriend00 Mar 21 '23 at 21:25
  • 1
    One might also ask why you're structing the code to have so many levels of hierarchy? If the code is structured into modules and modules are shared in a common location via either code structuring or a build process, then you `require()` in a module from the public location and that's all you have to do. Think about how NPM modules work in this regard. You don't have to go 5 levels deep to import code from an NPM module. So, I think the real issue here is how the code is structured, not how the tools work. – jfriend00 Mar 21 '23 at 21:27
  • 1
    Hi @jfriend00, `[FYI, none of the "Related Posts" in your question seem to have links...]` I originally had links in the "Related Posts" section, but Stackoverflow spam filter required me to remove the links before posting. – howard_9 Mar 21 '23 at 21:27
  • 1
    Hi @jfriend00, `[You seem to think this should be a feature of VS Code or webStorm. That is not simply the case....]` My PM seems to think that NodeJs, being the leading backend language, should have a way to avoid the ugly "../../../../../" style of referencing. He even said I am allowed to use any IDE (doesn't have to be VS Code or WebStorm), and I am allowed to modify package.json if needed (as long as no external packages are used). I found some interesting articles outside of StackOverflow that might lead to a solution...I will keep you guys posted. Thanks! – howard_9 Mar 21 '23 at 21:37
  • Did you try to use the absolute path? Having a `package.json` and starting your paths with a `/` should result in paths being relative to either root of `package.json` or the running file (do not remember which). Path like `/src/source.js` – Akxe Mar 21 '23 at 21:38
  • @Akxe — `/src/source.js` looks for a directory called `src` in the **root of the filesystem**, not the package. – Quentin Mar 21 '23 at 21:44
  • 1
    Hi @jfriend00, `[One might also ask why you're structing the code to have so many levels of hierarchy...]`, Let's say we have an international website that deals with cities and districts. So, we have a "United States" directory that contains a "California" folder, that contains a "Los Angeles" folder, which contains "Hollywood.js", etc. I wasn't aware that NodeJS only allowed 1-level deep structures. – howard_9 Mar 21 '23 at 21:49
  • 1
    Nodejs allows as many levels as you want. You just have to specify the path (absolute or relative) if you're not going to use a shared module architecture. Why do you have code in separate directories for every state, city, etc... Why wouldn't this be data in a database? I still think you have the tail wagging the dog here. Your chosen file structure is hard to deal with. So, perhaps fix the structure to be something that is easy to work with. – jfriend00 Mar 21 '23 at 21:54
  • 1
    Hi @Akxe, `[Did you try to use the absolute path]` Yes, I have tried "/src/source.js" and "src/source.js" and "./src/source.js". They failed with error "Cannot find module '/src/subFolder1/SubWorker.js'". – howard_9 Mar 21 '23 at 21:54
  • 1
    It sounds like you're taking direction from someone who doesn't know the capabilities or architecture of the tools and isn't aware that restructuring the layout of the files could make things a lot easier. That doesn't sound like an ideal situation. So, perhaps you should consider a restructuring of how the files are laid out as a good solution for the chosen tools. – jfriend00 Mar 21 '23 at 21:56
  • 1
    Hi @jfriend00, `[Nodejs allows as many levels as you want. You just have to specify the path (absolute or relative)]`, Databases stores data instead of programming logic. Interestingly, my project actually has programming logic associated with each city. =) – howard_9 Mar 21 '23 at 21:58
  • 1
    @howard_9 - That sounds like something that should be data-driven (generic code that uses data for each city) instead of custom code for each city. But, you appear not interested in my restructuring suggestions so I'll bow out now. – jfriend00 Mar 21 '23 at 22:01
  • 1
    Hi @jfriend00, `[It sounds like you're taking direction from someone who doesn't know the capabilities or architecture of the tools ]` Thanks for your feedback. I will definitely consider restructuring. On the other hand, since I have already started this post, it is still interesting to know whether there is a way to have "project root" referencing syntax in NodeJS. – howard_9 Mar 21 '23 at 22:02
  • I'm certain that you get node.js to resolve these paths to what you want. However, this would likely be achieved through code, not by configuration, and as such an IDE won't be able to use the same resolution mechanism - without a plugin at least. – Bergi Mar 21 '23 at 23:08
  • 1
    Hi @Bergi, `[I'm certain that you get node.js to resolve these paths to what you want...]` I just realized that NodeJS failed when I ran it in Windows cmd.exe or in Ubuntu terminal, so the failures had nothing to do with IDE...How would you solve this problem outside of an IDE? – howard_9 Mar 22 '23 at 00:32
  • 1
    @howard_9 I don't know how exactly this would work, I only know that it is possible to hook into the nodejs module resolution algorithm (if only by just monkeypatching `require`) so you will be able to achieve this with some hack. All I'm saying is that your IDE probably will not understand that hack, failing your requirement #3. – Bergi Mar 22 '23 at 00:38

4 Answers4

9

Using the "Imports" property

As of March 2023, a good way to eliminate the NodeJS relative paths is to use the imports property in package.json. (In the codes below, #root is the project root.)


For CommonJS-style JavaScripts:

// package.json
{
  "imports": {
    "#root/*.js": "./*.js"
  }
}

// main.js:
const Source = require('#root/path/to/Source.js');

// Source.js:
module.exports = class Source {
  // ...
}


For ECMAScript-style JavaScripts:

// package.json:
{
  "type" : "module",
  "imports": {
    "#root/*.js": "./*.js"
  }
}

// main.js
import { Source } from '#root/path/to/Source.js';

// Source.js:
export class Source {
  // ...
}

Advantages:

  • No need to "import" or "require" any additional packages (No Babel.js, No Webpack, No RequireJS). After installing NodeJS, this method works out of the box.

  • IDE linkages work as expected (Ctrl-click a class name to jump directly to the source file. Also, moving the source file (by drag and drop) will automatically update all references to the file. Tested on WebStorm 2022.3.2 and VS Code 1.76.2.)

  • Works with both .mjs (ECMAScript module system) and .cjs (CommonJS) file types. Please see this reference Post on .cjs and .mjs.

  • No need to modify the reserved node_modules directory

  • No need to set up any linux file links at the OS level

Updates in April 2023:

Arial
  • 326
  • 2
  • 11
1

Well, without violating your requirements, you can set the NODE_PATH environment variable to the root directory of your project.

In your absolute path of your project, locate the main.js file and add this for testing purpose to see your project root path:

const path = require('path');
const projectRoot = path.resolve(__dirname);
console.log('Project Root:', projectRoot);

Now set the NODE_PATH environment variable to the project root before running your application. You can do this in your terminal:

export NODE_PATH=/absolute/path/to/your/project/root

Replace /absolute/path/to/your/project/root with the path you got from the first step.

After that, you need to update the require in main.js:

const Worker = require('src/a1/a2/a3/Worker.js');
Worker.doWork();

And your Worker.js

const SubWorker = require('subFolder1/SubWorker.js');

module.exports =
class Worker { 
   static doWork() {
       console.log("doing work");
       SubWorker.doMoreWork();
   }
}

However, note that the NODE_PATH environment variable is deprecated since Node.js v16.9.0 and might be removed in future versions of Node.js. It is still supported in Node.js v16.x, but using it is not recommended for new projects. For future-proof solutions, you might want to consider using either ECMAScript modules with import maps or bundlers like webpack or rollup. You can read more about in the docs.

  • 1
    Thanks for your insights. Our company is using NodeJS v18.0, and I don't think I can downgrade to v16.0. – howard_9 Mar 22 '23 at 00:29
  • You can try without NODE_PATH, simple make a project root declartion `const path = require('path'); const projectRoot = path.dirname(__dirname);` Then you can import Worker.js like this `const Worker = require(path.join(projectRoot, 'src/a1/a2/a3/Worker.js')); Worker.doWork();` and for Worker.js `const path = require('path'); const SubWorker = require(path.join(__dirname, '../subFolder1/SubWorker.js'));` – Fadi Chamoun Mar 22 '23 at 01:58
  • I just tried "const projectRoot = path.resolve(__dirname);" in Worker.js. The value of "__dirname" in Worker.js is "./src/a1/a2/a3", not the project root. – howard_9 Mar 22 '23 at 15:13
0

You can try by using require.resolve method that returns a String with the resolved path. You can then use this path to import the file using require.

As per your code structure, this should work for you in Worker.js file:

const resolvedPath = require.resolve('./SubWorker.js', { paths:['subFolder1'] }) 
const SubWorker = require(resolvedPath);

A point to note over here is that this approach has processing overhead.

Update:

The code works fine in WebStorm with the exact same folder structure as yours:

enter image description here

Also, you can see it working on StackBlitz

Prerak Sola
  • 9,517
  • 7
  • 36
  • 67
  • 1
    Hi @Prerak_Sola, I just tried this in my setup (using WebStorm 2022.3.2 and VS Code 1.76.2), and I got this error: "Error: Cannot find module 'subFolder1/SubWorker.js'" – howard_9 Mar 21 '23 at 22:22
  • By the way, I have tried the "path.resolve" solution in my Attempt #1 above. – howard_9 Mar 21 '23 at 22:25
  • @howard_9 It seems to work fine in WebStorm. Check the updated answer with the screenshot. – Prerak Sola Mar 21 '23 at 22:34
  • Here's a working example of it https://stackblitz.com/edit/node-i7s4uf?file=main.js – Prerak Sola Mar 21 '23 at 22:43
  • 1
    Indeed, this solution works on Macbooks and stackblitz. Stackblitz uses NodeJS v16.14.2, so I installed v16.14.2 on my Ubuntu...and, it failed on my Ubuntu. (Error messages are shown below). Also, I checked the results on my Windows 10(v22H2) using NodeJS v18.15.0. The same error messages appear. Therefore, I am hoping more people here can confirm the results on their environments. So far, this method seems to work only on Macbooks and stackblitz, but not on Ubuntu and Windows. [Error_Screenshot](https://ibb.co/YLrHR7H) (In both Windows and Ubuntu, relative paths still work.) – howard_9 Mar 22 '23 at 00:22
0

look at these 2 npm packages, one of them should help you:

1.app-root-path (https://www.npmjs.com/package/app-root-path)

  1. module-alias (https://www.npmjs.com/package/module-alias)
  • 2
    I am unable to install anything due to Requirement #1 – howard_9 Mar 22 '23 at 14:39
  • 1
    Ok...I upvoted this answer, because app-root-path and module-alias work in command line mode. I haven't chosen this as the accepted answer yet, because in IDE (WebStorm and VS Code), the linkages don't work anymore after installing module-alias. (Requirement #3) – howard_9 Mar 22 '23 at 17:21
  • I downvoted this answer because it does not show how to use these packages in a way that answers the question; it is essentially "your answer is over there; figure it out". – Heretic Monkey Mar 29 '23 at 23:41