5

Our app consists of two components: the API and the client. Both are independent Node applications. While the API doesn't really have any frontend components besides the docs, they both still share some common files, like Jade components, normalizing CSS, utility modules and most importantly Mongoose's schema & model definitions for MongoDB interaction. I really don't want to get used to a workflow where I first make changes to the API, and then copy the changed files to the client or vice versa, so it would be great to find a viable solution for this.

What would be the best way to share miscellaneous code across Node apps?

While browsing the web, I've come across a couple of solutions, but none of them really work in this case.

One solution is to make a node module out of the common files and keep it in sync between the apps with NPM, but that would mean I (and possible future devs) have to know which files are part of the common module, and require them with require('node_modules/mongo/schemas/example.js') instead of require('mongo/schemas/example.js') for example. Since hypothetically there might be hundreds of common files, it would be impossible to remember if a file is common or not, especially for new devs.

Another solution is to make a git submodule of the common module. This would be perfect, if only I could include the submodule path to Node's require paths, so in case it didn't find the required file in the expected location (say, mongo/schemas/example.js), it would look in the common submodule (common/mongo/schemas/example.js). This would be awesome, since it would allow overwriting a common file with a "local" version. It's kind of like Magento works. The problem here is that Node doesn't allow editing the require paths (anymore at least, supposedly it was possible prior to Node 0.5), so that's a bummer.

The third solution is to brutally merge the common repository's contents into the app's root. There are no truly insurmountable problems in this solution, but I just feel like it's a bad practice. After all, it would make it impossible to edit the common files through the app, since there's no way to selectively push the common files into the common git repository. So basically this solution would only make the workflow even more tedious: all modifications to the common files would have to be copied by hand to the common codebase, pushed to the git remote and then copied to the other app. So this solution definitely doesn't cut it for me either.

It seems that the second solution is the only (and best) option I have, unless you guys come up with a better one (which I'm really hoping for :D). If anyone knows a way to change node's require paths, I would be extremely grateful.

Aapo Kiiso
  • 166
  • 3
  • 12
  • Excellent question. I heard a talk by someone that you should put all your code in npmjs.org. But that's only feasible if you can open source it. You can make a private npm repo, but I've never done it. http://stackoverflow.com/questions/7575627/can-you-host-a-private-repository-for-your-organization-to-use-with-npm – Jess Feb 01 '14 at 02:02

3 Answers3

3

One solution is to make a node module out of the common files and keep it in sync between the apps with NPM

It's the only one good solution. If you have a shared code, it should be a module.

but that would mean I (and possible future devs) have to know which files are part of the common module, and require them with require('node_modules/mongo/schemas/example.js') instead of require('mongo/schemas/example.js') for example

No. It's require('mongo/schemas/example.js') instead of require('./mongo/schemas/example.js'). Or better yet, require('mongo').Schemas.example, though it requires some work.

You have to split your codebase into a logical modules. I guess that kind of a nightmare you have there would justify use of NODE_PATH, but I'd really suggest to refactor that...

alex
  • 11,935
  • 3
  • 30
  • 42
  • 2
    +1 Although worth mentioning that if you have private modules you may also use a git repo (on github for example) to host a module. Coupled with git tags you still get semantic versioning. See http://www.devthought.com/2012/02/17/npm-tricks/ – qubyte Jan 31 '14 at 23:38
  • @MarkS.Everitt, unfortunately, you won't get semantic versioning with git tags, npm [don't support it](https://github.com/npm/npm/issues/4527). You'll have to use either [yapm](https://github.com/rlidwka/yapm) or [visionmedia/npm](https://github.com/visionmedia/npm) to get it working. – alex Feb 01 '14 at 00:46
  • PS: advice is however right, you can use either private registry or github repository to do this – alex Feb 01 '14 at 00:47
  • 1
    Ups, you are right of course. That's what I get for playing with TJ's npm fork. Git tags do work with regular npm, but are pinned. – qubyte Feb 01 '14 at 00:49
  • While this could work for the mongo schemas and whatnot, how would you deal with the static assets, like `/js/index.js` or something? Should they also be placed in modules, for example `[app_name]-assets/js/index.js`? I have a feeling that wouldn't work very well with Express' static file handling. – Aapo Kiiso Feb 01 '14 at 14:00
  • Yes, static files are placed within modules that export small express applications serving static. It's actually an easy part. – alex Feb 01 '14 at 14:12
2

I tried to find a solution for your exact same problem and did all the things I pointed out in your other question. Everything was working ok but I knew my workflow could be improved.

Attempt 1: add remote dependencies to package.json and load as npm dependencies

Didn't work because when Heroku installs dependencies upon deployment it doesn't have the ssh key to pull from github and basic auth (exposed username and password) was not an option.

Attempt 2: git submodules to the rescue

This brings a bunch of problems already discussed in your other question.

I finally ended up with ONE (well thought out) PROJECT THAT HAS EVERYTHING.

I run my app with a flag:

node app --module api

or

node app --module manager

and load dependencies accordingly.

It might not suit every project but if you have models,views,templates,constants and more business logic shared across applications, front-end and whatnot the trade off is pretty good. I don't think storage is a problem nowadays so installing a few more dependencies and unused modules might not be a problem (again,it depends on the size of the project(s) and amount of dependencies, in my case it wasn't a problem at all).

You then can target app specific tasks per service with grunt:

grunt deploy:API, grunt deploy:manager, grunt:build:manager, etc ...

Community
  • 1
  • 1
Maroshii
  • 3,937
  • 4
  • 23
  • 29
  • Thanks for your input, I'll definitely look into this! – Aapo Kiiso Feb 01 '14 at 14:04
  • Sorry, I'm a bit late with my question, but I'm facing the same problem at the moment. How do you manage the "--module" parameter? Is there any configuration option in the package.json file for the module parameter or do you simply use the process.argv parameter in app.js to detect which module you have to load/run? – Nrgyzer Jan 27 '18 at 08:00
  • In this case the `--module` parameter is handled by the app.js file (`node app ...` implicitly runs the app.js file for current working directory). Your `app.js` should be the sole entrypoint, parse the command line flags and import/run modules based on this. You can use something like: https://www.npmjs.com/package/yargs to parse the arguments and start the application in whichever way you like. Hope this helps! – Maroshii Feb 05 '18 at 22:20
0

Excellent question.

"but that would mean I (and possible future devs) have to know which files are part of the common module, and require them with require('node_modules/mongo/schemas/example.js')"

This is actually not true because require will search the project directory tree for a node_modules folder as per the Loading from node_modules Folders doc.

So your first option to use npm is a very good one. I have made a module in npm too actually. Once you set up the package.json, you can use npm version and npm publish to publish to npmjs.org. Then you can put the desired packages in your other project(s) package.json and download the required versions from npm using npm install.

One problem is that you must open source your code on npmjs.org, so if you don't want to, you need to create your own private npm repo.

Cheers.

Community
  • 1
  • 1
Jess
  • 23,901
  • 21
  • 124
  • 145
  • 1
    Yeah, the node_modules example was kind of misleading, since it doesn't describe the whole problem. I don't think I can store static files, like JS or CSS very efficiently with node modules, since they are handled out of the /public directory, and I'm not exactly sure how I would route the static assets out of node_modules without messsing the app's local asset routing. Also, our code is private so npmjs.org isn't an option, and making our own npm repo seems to be a bit of a project itself. – Aapo Kiiso Feb 01 '14 at 14:07
  • For that `require`ment :D you could try **browserify** at http://browserify.org/. – Jess Feb 03 '14 at 13:41