Coverage-based fuzzing of JavaScript applications

Question

The American Fuzzy Lop, and the conceptually related LLVM libfuzzer not only generate random fuzzy strings, but they also watch branch coverage of the code under test and use genetic algorithms to try to cover as many branches as possible. This increases the hit frequency of the more interesting code further downstream as otherwise most of the generated inputs will be stopped early in some deserialization or validation.

But those tools work at native code level, which is not useful for JavaScript applications as it would be trying to cover the interpreter, but not really the interpreted code.

So is there a way to fuzz JavaScript (preferably in browser, but tests running in node.js would help too) with coverage guidance?

I looked at the tools mentioned in this old question, but those that do javascript don't seem to mention anything about coverage profiling. And while radamsa mentions optionally pairing it with coverage analsysis, I haven't found any documentation on how to actually do it.

How can one fuzz-test java-script (in browser) application with coverage guidance?

Answer 1

Fuzzing a JavaScript engine draws a lot of attention as the number of browser users is about 4 Billion. Several works have been done to find bugs in JS engines, including popular large engines, e.g, v8, webkit, chakracore, gecko, or some small embedded engines, like jerryscript, QuickJS, jsish, mjs, mujs.

It is really difficult to find bugs using AFL as the mutation mechanisms provided by AFL is not practical for JS files, e.g, bitflip can hardly be a valid mutation. Since JS is a structured language, several works using ECMAScript grammar to mutate/generate JS files(seeds):

LangFuzz parses sample JS files and splits them into code fragments. It then recombines the fragments to produce test cases.

jsfunfuzz randomly generates syntactically valid JS statements from JS grammar manually written for fuzzing.

Dharma is a generation-based, context-free grammar fuzzer, generating files based on given grammar.

Superion extends AFL using tree-based mutation guided by JS grammar.

The above works can easily pass the syntax checks but fail at semantic checks. A lot of generated JS seeds are semantically invalid.

CodeAlchemist uses a semantics-aware approach to generate code segments based on a static type analysis.

There are two levels of bugs related to JS engines: simple parser/interpreter bugs and deep inside logic bugs. Recently, there is a trend that the number of simple bugs decreases while more and more deep bugs come out.

DIE uses aspect-preserving mutation to preserves the desirable properties of CVEs. It also using type analysis to generate semantic-valid bugs.

Some works focus on mutating intermediate representations.

Fuzzilli is a coverage-guided fuzzer based on mutation on the IR level. The mutations on IR can guarantee semantic validity and can be transferred to JS.

Fuzzing JS is an interesting and hot topic according to the top conference of security/SE in recent years. Hope this information is helpful.