11

While working on a larger web application with an increasing amount of JavaScript code, we did a brainstorming session on how to improve code quality.

One of the first ideas was to introduce unit tests. This will be a long term goal; that will not, however, fix the most common causes of regression: the changing DOM and browser specific issues.

Unit tests run in a mocked, DOM-less environment and are not on the page.

What I'm looking for is an assertion framework that can be plugged into the code like this:

var $div = $("div.fooBarClass");
assertNotEmpty($div);
$div.fooBarAction();

I've found assertion frameworks that can do this, but they all either log into the console or into the DOM or open a silly pop-up. None of these work together with (thousands of) automated tests. What I'm looking for is a run-time assertion framework that logs failed assertion via AJAX! Ideally, it should be:

  • Have common assertions built-in.
  • Integrate with JQuery modules, closures.
  • Log (via Ajax) the assertion, the file name, the page, line number, the cause of failure, some pre-configured variables of the environment (browser, release version etc.).
  • Support callbacks in case of failures. (If any assertion framework can just do this, I would be gladly willing to write callbacks doing the Ajax part.)
  • Work well with all browsers.
  • Trivial to exclude from production release.
  • Maintained code base.
Community
  • 1
  • 1
sibidiba
  • 6,270
  • 7
  • 40
  • 50

3 Answers3

2

We've been using the YUI Test Library. It seems to work fairly well.

Has a variety of assertion methods for different types

Assertions exist for equality, sameness, true, false, object type, and even array item comparison.

Allows for mock objects to test DOM objects and other functions Our code does a lot of AJAX calls, or requires methods / objects that don't need to be tested (as they are tested elsewhere). Using Mock objects, we can tell the tests what to expect. For example:

var mockXhr = Y.Mock();

//I expect the open() method to be called with the given arguments
Y.Mock.expect(mockXhr, {
    method: "open",
    args: ["get", "/log.php?msg=hi", true]                            
});

Works with all browsers

We run our tests in IE, Chrome, and Firefox, and aside from some differences in what the test runner itself looks like, it works!

Trivial to exclude from production release

We have all of our testing code in a separate folder which accesses all the production code. Excluding the tests from production is as easy as excluding a folder.

Maintained codebase

YUI 3 is used on the Yahoo homepage, and seems to be fairly well maintained.

NT3RP
  • 15,262
  • 9
  • 61
  • 97
  • What does happen if an assertion fails? Can it report via AJAX? I do not need unit tests for the modules themselves. I want to make assertions in the QA environment while the whole webapp is running (and exclude/noop in production). An example: I want to assert on the presence of DOM elements. – sibidiba Mar 03 '11 at 16:20
  • What do you mean when you say 'report via AJAX'? If an assertion fails, then the test case fails, and the test suite continues on with other tests. – NT3RP Mar 03 '11 at 16:21
  • I do not want a test suite. I want to insert assertions into the code itself, not unit tests. For example a common problem is that the UI team is changing the DOM and as a consequence some JS logic starts failing. Unit tests do not cover this, because the JS module works perfectly. I want to insert assert calls into the normal code flow, and report failures via AJAX so they are reported while standard automated or manual tests are run. – sibidiba Mar 03 '11 at 16:23
  • 1
    Hmm. Well, it seems I misunderstood your question. I think you might be best off writing your own in this case, though I know that's not the answer you're looking for. – NT3RP Mar 03 '11 at 16:59
0

I know that it is not what you asked for but I highly recommend Selenium for automated testing of web applications.

  • Common assertions are built in.
  • It can test any JS framework because it drives the browser where your code runs.
  • It has robust logging features.
  • Browser support depends on your OS but all major browsers are supported.
  • There is nothing to exclude from a production release because the tests are external to the application.
  • The code base is well maintained and you have full control over your test cases.
Mr. Muskrat
  • 22,772
  • 3
  • 20
  • 21
  • We exactly use selenium right now. But those are functional tests. We want to include assertions to catch glitches during those tests. It is not possible to cover all our functionality in Selenium, we already have thousands of tests. But assertions could catch those JS errors that do not trigger selenium test failures. – sibidiba Feb 15 '11 at 16:53
  • 1
    Like I said, it wasn't what you asked for but it might help someone else. – Mr. Muskrat Feb 15 '11 at 17:02
0

It seems there is no similar solution I'm looking for.

I'm going write my own one, overriding console.assert to make an ajax call when arguments evaluate to false.

UPDATE: Here it comes, still under development, https://github.com/gaboom/qassert

sibidiba
  • 6,270
  • 7
  • 40
  • 50