Generate Ember code coverage for a Selenium test run

Question

Does anyone know how to generate Ember code coverage metrics off of Selenium tests? I've got several tests written in Groovy/Java.

I've found documentation of people doing this with Istanbul. One example is Front End Javascript Test Coverage with Istanbul Selenium.

There is also an Ember CLI Code Coverage tool that builds upon the instrumentation produced by Istanbul. However, from what I've gathered so far, this tool appears to completely wrap around the Istanbul instrumentation, thus only allowing code coverage statistics to be generated for ember unit and integration tests before build time. Where it stores its own mappings from the Istanbul results to Ember code is not readily apparent, at least not to a Java developer like me.

I'm looking for a way to generate a war file for internal use only that has both the Istanbul instrumentation and the mappings to the ember code, and that Selenium can trigger to generate the Ember Code Coverage report. Any insight on how to do this would be much appreciated.

Answer 1

Turns out that the main obstacle that the above sights did not address was not Ember itself, which is basically just Javascript. The problem lies in the fact that Ember is based on the newer ES6 version of Javascript, and the original version of Istanbul was written for ES5.

For ES6 instrumentation, the application Babel, which is used for cross browser compatibility, has an Istanbul plugin available. This Istanbul tutorial shows how to use it for unit testing. The trick is figuring out how to channel the instrumentation process to get the instrumented code into a war file that can then be dispatched to a server that a browser can hit. (Our Ember experts figured that one out, so I can't outline that solution in this response.)

Once a server is running with that war file where your Selenium tests can hit it, you have to add code to the end of your tests (right before they leave that site) to access the window.__coverage__ object and output it to file at the end of each test session. For a sample of how to do that, see this answer to a similar question: https://sqa.stackexchange.com/a/41715/29877.

Say you write that code to output one or more coverage files to a the directory named "usage" and create another directory named "coverage" beside it. You can then run this command from their common parent directory to generate a basic html report in the "coverage" directory:

nyc report --exclude-after-remap=false --report-dir=./coverage --reporter=html -t ./usage

(See these instructions to install the nyc utility.)

Now you can view that report by opening up coverage/index.html in a browser. Of course, nyc has several --reporter options for producing other types of reports.