Load "Vanilla" Javascript Libraries into Node.js

Question

There are some third party Javascript libraries that have some functionality I would like to use in a Node.js server. (Specifically I want to use a QuadTree javascript library that I found.) But these libraries are just straightforward .js files and not "Node.js libraries".

As such, these libraries don't follow the exports.var_name syntax that Node.js expects for its modules. As far as I understand that means when you do module = require('module_name'); or module = require('./path/to/file.js'); you'll end up with a module with no publicly accessible functions, etc.

My question then is "How do I load an arbitrary javascript file into Node.js such that I can utilize its functionality without having to rewrite it so that it does do exports?"

I'm very new to Node.js so please let me know if there is some glaring hole in my understanding of how it works.

---

EDIT: Researching into things more and I now see that the module loading pattern that Node.js uses is actually part of a recently developed standard for loading Javascript libraries called CommonJS. It says this right on the module doc page for Node.js, but I missed that until now.

It may end up being that the answer to my question is "wait until your library's authors get around to writing a CommonJS interface or do it your damn self."

Answer 1

Here's what I think is the 'rightest' answer for this situation.

Say you have a script file called quadtree.js.

You should build a custom node_module that has this sort of directory structure...

./node_modules/quadtree/quadtree-lib/
./node_modules/quadtree/quadtree-lib/quadtree.js
./node_modules/quadtree/quadtree-lib/README
./node_modules/quadtree/quadtree-lib/some-other-crap.js
./node_modules/quadtree/index.js

Everything in your ./node_modules/quadtree/quadtree-lib/ directory are files from your 3rd party library.

Then your ./node_modules/quadtree/index.js file will just load that library from the filesystem and do the work of exporting things properly.

var fs = require('fs');

// Read and eval library
filedata = fs.readFileSync('./node_modules/quadtree/quadtree-lib/quadtree.js','utf8');
eval(filedata);

/* The quadtree.js file defines a class 'QuadTree' which is all we want to export */

exports.QuadTree = QuadTree

Now you can use your quadtree module like any other node module...

var qt = require('quadtree');
qt.QuadTree();

---

I like this method because there's no need to go changing any of the source code of your 3rd party library--so it's easier to maintain. All you need to do on upgrade is look at their source code and ensure that you are still exporting the proper objects.

Answer 2

There is a much better method than using eval: the vm module.

For example, here is my execfile module, which evaluates the script at path in either context or the global context:

var vm = require("vm");
var fs = require("fs");
module.exports = function(path, context) {
  context = context || {};
  var data = fs.readFileSync(path);
  vm.runInNewContext(data, context, path);
  return context;
}

And it can be used like this:

> var execfile = require("execfile");
> // `someGlobal` will be a global variable while the script runs
> var context = execfile("example.js", { someGlobal: 42 });
> // And `getSomeGlobal` defined in the script is available on `context`:
> context.getSomeGlobal()
42
> context.someGlobal = 16
> context.getSomeGlobal()
16

Where example.js contains:

function getSomeGlobal() {
    return someGlobal;
}

The big advantage of this method is that you've got complete control over the global variables in the executed script: you can pass in custom globals (via context), and all the globals created by the script will be added to context. Debugging is also easier because syntax errors and the like will be reported with the correct file name.

Answer 3

The simplest way is: eval(require('fs').readFileSync('./path/to/file.js', 'utf8')); This works great for testing in the interactive shell.

Answer 4

AFAIK, that is indeed how modules must be loaded. However, instead of tacking all exported functions onto the exports object, you can also tack them onto this (what would otherwise be the global object).

So, if you want to keep the other libraries compatible, you can do this:

this.quadTree = function () {
  // the function's code
};

or, when the external library already has its own namespace, e.g. jQuery (not that you can use that in a server-side environment):

this.jQuery = jQuery;

In a non-Node environment, this would resolve to the global object, thus making it a global variable... which it already was. So it shouldn't break anything.

Edit: James Herdman has a nice writeup about node.js for beginners, which also mentions this.

Answer 5

I'm not sure if I'll actually end up using this because it's a rather hacky solution, but one way around this is to build a little mini-module importer like this...

In the file ./node_modules/vanilla.js:

var fs = require('fs');

exports.require = function(path,names_to_export) {
    filedata = fs.readFileSync(path,'utf8');
    eval(filedata);
    exported_obj = {};
    for (i in names_to_export) {
        to_eval = 'exported_obj[names_to_export[i]] = ' 
            + names_to_export[i] + ';'
        eval(to_eval); 
    }
    return exported_obj;
}

Then when you want to use your library's functionality you'll need to manually choose which names to export.

So for a library like the file ./lib/mylibrary.js...

function Foo() { //Do something... }
biz = "Blah blah";
var bar = {'baz':'filler'};

When you want to use its functionality in your Node.js code...

var vanilla = require('vanilla');
var mylibrary = vanilla.require('./lib/mylibrary.js',['biz','Foo'])
mylibrary.Foo // <-- this is Foo()
mylibrary.biz // <-- this is "Blah blah"
mylibrary.bar // <-- this is undefined (because we didn't export it)

Don't know how well this would all work in practice though.

Answer 6

I was able to make it work by updating their script, very easily, simply adding module.exports = where appropriate...

For example, I took their file and I copied to './libs/apprise.js'. Then where it starts with

function apprise(string, args, callback){

I assigned the function to module.exports = thus:

module.exports = function(string, args, callback){

Thus I'm able to import the library into my code like this:

window.apprise = require('./libs/apprise.js');

And I was good to go. YMMV, this was with webpack.

Answer 7

A simple include(filename) function with better error messaging (stack, filename etc.) for eval, in case of errors:

var fs = require('fs');
// circumvent nodejs/v8 "bug":
// https://github.com/PythonJS/PythonJS/issues/111
// http://perfectionkills.com/global-eval-what-are-the-options/
// e.g. a "function test() {}" will be undefined, but "test = function() {}" will exist
var globalEval = (function() {
    var isIndirectEvalGlobal = (function(original, Object) {
        try {
            // Does `Object` resolve to a local variable, or to a global, built-in `Object`,
            // reference to which we passed as a first argument?
            return (1, eval)('Object') === original;
        } catch (err) {
            // if indirect eval errors out (as allowed per ES3), then just bail out with `false`
            return false;
        }
    })(Object, 123);
    if (isIndirectEvalGlobal) {
        // if indirect eval executes code globally, use it
        return function(expression) {
            return (1, eval)(expression);
        };
    } else if (typeof window.execScript !== 'undefined') {
        // if `window.execScript exists`, use it
        return function(expression) {
            return window.execScript(expression);
        };
    }
    // otherwise, globalEval is `undefined` since nothing is returned
})();

function include(filename) {
    file_contents = fs.readFileSync(filename, "utf8");
    try {
        //console.log(file_contents);
        globalEval(file_contents);
    } catch (e) {
        e.fileName = filename;
        keys = ["columnNumber", "fileName", "lineNumber", "message", "name", "stack"]
        for (key in keys) {
            k = keys[key];
            console.log(k, " = ", e[k])
        }
        fo = e;
        //throw new Error("include failed");
    }
}

But it even gets dirtier with nodejs: you need to specify this:

export NODE_MODULE_CONTEXTS=1
nodejs tmp.js

Otherwise you cannot use global variables in files included with include(...).