Static-analysis with custom rules for JavaScript?

Question

Does JSLint, JSHint, or some other open-source static code analysis tool support adding custom rules for code compliance, or are there some ECMAScript compliant parsers that I can use to get the results as close as possible to the ones seen in the snippet below?

For example, I’d like to look into JavaScript code and list what functions are called, if it calls a library (or APIs provided by smartphones for HTML5 widgets) to register all that fall under the namespaces of that API, to make a tree of the objects and their properties to see if function is called out from what object can be traced back to, maybe with an output in XML, JSON or other structured format.

Say for example I have this JavaScript code (it does nothing and is just for the sake of the argument):

jobs = mylibrary.getJobs();
found = jobs.find("Python");
list = found.convert("html");

I want my analyzer tool to get this:

{
    "mylibrary": {
        "jobs": {"maker":"getJobs", "parent": "mylibrary"},
        "found": {"maker": "find", "parent": "jobs", "parameters": "Python"},
        "list": {"maker": "convert", "parent": "found"}
    }
}

Do you mean like analytics but returning XML data of Analytics on your HTML5 content? Example, monitor all the people clicking HTML5 audio and then send that data in XML data so you can export to charts etc? — TheBlackBenzKid, Aug 06 '12 at 09:30
@TheBlackBenzKid: I mean static-analysis, (not a dynamic-analysis, ) but on code (both .js and — Eduard Florinescu, Aug 06 '12 at 10:42
@TheBlackBenzKid: That would be still dynamic-analysis, what I need in fact is something that generates a DOM-like structure (maybe as an XML or Jason) but only from the javascript objects without running the .js through a browser engine. Thanks for the reply it makes me think more clear what I need. — Eduard Florinescu, Aug 06 '12 at 16:14

Linus Thiel · Accepted Answer · 2012-09-10T15:20:21.280

You should be able to build something like this using substack's burrito, which uses the parser from Uglify-JS and gives, I think, you all you need. A quick sample:

src.js:

var jobs, found, list;
jobs = mylibrary.getJobs();
found = jobs.find("Python");
list = found.convert("html");

ast.js:

var fs = require('fs'),
    burrito = require('burrito');

var src = fs.readFileSync('./src.js', 'utf8');

burrito(src, function (node) {
    console.log(node.name, node.value);
});

Exactly how you would build your requested structure, I'm not too sure (I'm not very well versed in AST parsing myself) but I'm sure it would entail some effort on your part. Perhaps you wouldn't need a structure in-between, so to speak, but could just validate each node from burrito, where each call node would be validated against it's values (function name, object name etc), with a warning raised if it doesn't validate.

Here is the output from the burrito call above (note: every [Object] or such has been truncated by node.js' console.log. Values are actually nodes in the parse tree from burrito, so each value has it's associated state etc).

var [ [ [ 'jobs' ], [ 'found' ], [ 'list' ] ] ]
stat [ [ { name: 'assign', start: [Object], end: [Object] },
    true,
    [ [Object], 'jobs' ],
    [ [Object], [Object], [] ] ] ]
assign [ true,
  [ { name: 'name', start: [Object], end: [Object] }, 'jobs' ],
  [ { name: 'call', start: [Object], end: [Object] },
    [ 'dot', [Object], 'getJobs' ],
    [] ] ]
name [ 'jobs' ]
call [ [ 'dot', [ 'name', 'mylibrary' ], 'getJobs' ], [] ]
stat [ [ { name: 'assign', start: [Object], end: [Object] },
    true,
    [ [Object], 'found' ],
    [ [Object], [Object], [Object] ] ] ]
assign [ true,
  [ { name: 'name', start: [Object], end: [Object] }, 'found' ],
  [ { name: 'call', start: [Object], end: [Object] },
    [ 'dot', [Object], 'find' ],
    [ [Object] ] ] ]
name [ 'found' ]
call [ [ 'dot', [ 'name', 'jobs' ], 'find' ],
  [ [ [Object], 'Python' ] ] ]
string [ 'Python' ]
stat [ [ { name: 'assign', start: [Object], end: [Object] },
    true,
    [ [Object], 'list' ],
    [ [Object], [Object], [Object] ] ] ]
assign [ true,
  [ { name: 'name', start: [Object], end: [Object] }, 'list' ],
  [ { name: 'call', start: [Object], end: [Object] },
    [ 'dot', [Object], 'convert' ],
    [ [Object] ] ] ]
name [ 'list' ]
call [ [ 'dot', [ 'name', 'found' ], 'convert' ],
  [ [ [Object], 'html' ] ] ]
string [ 'html' ]

Update:

Another option is the newer(?) ES parser Esprima, which seems to be both more actively developed and better documented. It's also reportedly faster than Uglify. You can try out e.g. parsing on the Parsing Demo page. You sould be able to build a good solution using this, methinks.

Thank you, this looks the most promising until now I will play around with it, to see what I can accomplish. — Eduard Florinescu, Sep 10 '12 at 13:37
Sounds good -- please see my note about Esprima, too -- it certainly seems handy! — Linus Thiel, Sep 10 '12 at 15:20
Esprima is even better, your answer is the best until now, for today I finished the votes and also I want to leave the question open until the end of the bounty ;) — Eduard Florinescu, Sep 10 '12 at 18:12

score 1 · Answer 2 · edited May 23 '17 at 12:04

I tried something with a javascript interpreter that can be accessed from code (in my case python). So interpreters like pynoceros, pynarcissus or pyv8 might help me.

There is an answer here on how to install py8: https://stackoverflow.com/a/11879224/1577343

Since with the above approach I didn't had much success I prefer a static analysis solution that uses a ECMAScript compliant parser.

With static analysis as far I could get is using JSLINT parser( Run JSLint on a .js file from debugging console in chrome or firefox): But I don't know how to use this further.

{
    "string": "(begin)",
    "first": [
        {
            "string": "var",
            "arity": "statement",
            "first": [
                {
                    "string": "jobs"
                },
                {
                    "string": "found"
                },
                {
                    "string": "list"
                }
            ]
        },
        {
            "string": "=",
            "arity": "infix",
            "first": {
                "string": "jobs"
            },
            "second": {
                "string": "(",
                "arity": "infix",
                "first": {
                    "string": ".",
                    "arity": "infix",
                    "first": {
                        "string": "mylibrary"
                    },
                    "second": {
                        "string": "getJobs"
                    }
                },
                "second": []
            }
        },
        {
            "string": "=",
            "arity": "infix",
            "first": {
                "string": "found"
            },
            "second": {
                "string": "(",
                "arity": "infix",
                "first": {
                    "string": ".",
                    "arity": "infix",
                    "first": {
                        "string": "jobs"
                    },
                    "second": {
                        "string": "find"
                    }
                },
                "second": [
                    {
                        "string": "Python",
                        "arity": "string"
                    }
                ]
            }
        },
        {
            "string": "=",
            "arity": "infix",
            "first": {
                "string": "list"
            },
            "second": {
                "string": "(",
                "arity": "infix",
                "first": {
                    "string": ".",
                    "arity": "infix",
                    "first": {
                        "string": "found"
                    },
                    "second": {
                        "string": "convert"
                    }
                },
                "second": [
                    {
                        "string": "html",
                        "arity": "string"
                    }
                ]
            }
        }
    ]
}

score 1 · Answer 3 · edited Jun 20 '20 at 09:12

PMD supports ECMAScript static analysis with custom rules:

Use one of the current rulesets as an example. Copy and paste it into your new file, delete all the old rules from it, and change the name and description.

Notice that you can customize individual referenced rules. Everything but the class of the rule can be overridden in your custom ruleset.

You can also exclude certain files from being processed by a ruleset using exclude patterns, with an optional overriding include pattern. A file will be excluded from processing when there is a matching exclude pattern, but no matching include pattern.

Path separators in the source file path are normalized to be the '/' character, so the same ruleset can be used on multiple platforms transparently.

Additionally, this exclude/include technique works regardless of how PMD is used (e.g. command line, IDE, Ant), making it easier to keep application of your PMD rules consistent throughout your environment.

You can specify the full path to your custom ruleset name alongside of the built-in PMD rulesets

To see it in your IDE, add it to rulesets/rulesets.properties

References

Static-analysis with custom rules for JavaScript?

3 Answers3

Linked