4

I'm using the meteor-coverage package (version 1.1.4) with mocha (version 2.4.5_6) and meteor version 1.4.4.1 on Ubuntu 14.04 LTS. I have been able to produce very pretty test coverage reports, but it seems that for the client-side tests something is amiss. In order to send the coverage data to localhost:3000/coverage I have created a function called sendCoverage() which I import in my .tests.js files:

export const sendCoverage = function sendCoverage() {
    Meteor.sendCoverage(function(stats,err) {console.log(stats,err);});
};

I call this function after a block of mocha tests:

after (function () {
    sendCoverage();
});

Now, this produces test coverage reports in my localhost:3000/coverage page, but it seems as though it does not display the coverage properly. For example, I see that some statements are executed, but are highlighted in red and flagged as not covered. For example:

coverage example

It seems as though the statements get executed 11 and 12 times, respectively. However, they are not flagged as being covered and in my reports the percentage of statement coverage reflects this.

Does anyone know what I may be doing wrong and/or have experience with client-side code coverage and the meteor-coverage package?

Thanks!

Post-solution edit

It seems I've got it working now. The percentages on Codacy match the percentages in my html report. Looking at the html report more closely, it seems the coverage numbers were correct after all. It was simply the drill down that was showing odd behavior. So, the conclusion is that it worked after all, but it took Codacy's second opinion to confirm this to me. My new approach will be to create lcov coverage reports with spacejam (see Ser's answer below) and export these to an external service like Codacy, Codecov or SonarQube.

Thanks Serut for the input!

pedrorijo91
  • 7,635
  • 9
  • 44
  • 82
Gijs
  • 84
  • 10

1 Answers1

1

I am the author of meteor-coverage. Glad to see that the package runs well on your app!

First, I don't think the way your are collecting coverage and saving reports is optimized: don't create an utils with a function to save the coverage. You can save the coverage on each file just by using (assuming Meteor.sendCoverage always exists on test). 

after (function () {
    Meteor.sendCoverage(()=>{});
});

On the other hand, you should not write any code in your test file in order to save coverage. The test runner can do that for you like what I added on the spacejam fork. You can try to export the html and the lcov report using serut/spacejam.

I think the lcov format is more reliable than the html report. If I look at some coverage report of the client side code from meteor-coverage, everything looks coherent. Try to send the lcov file to an Code Quality platform like Sonar, Codecov or Codacy. I hope it will fix the line problem, which may be related to istanbul and its html report generation.

Ser
  • 2,661
  • 23
  • 28
  • Hey Serut, thanks for your answer! I attempted to use spacejam to run the tests in the manner you explained. However, this led to the following error output in my console: `=> Started proxy. => Started MongoDB. spacejam: meteor mongodb is ready spacejam: killing meteor spacejam: Unknown error with exit code 'null'. Exiting.` I have tried to find anything about this online, but all issues related to this seem to deal with Windows support, which, since I am developing my application on Ubuntu, does not seem to be applicable to me. Do you know where this error might come from? – Gijs Jun 04 '17 at 09:47
  • The stacktrace is too short to let me found out what's going on, it can be the way you are launching `spacejam` (use `meteor npm run `) or a bug linked to your setup. Check [this fork I've made of apollographql/meteor-starter-kit](https://github.com/serut/meteor-react-apollo-test-coverage-starter-kit) to see changes I've made to add coverage. Or [fill an issue](https://github.com/serut/meteor-coverage/issues) if you want this to be fixed. – Ser Jun 04 '17 at 12:27
  • Alright, so I managed to get spacejam up and running. I did have to use the `https://github.com/serut/spacejam/tarball/windows-suppport-rc4` version after all. Was able to create an lcov file with the provided package.json commands in your repo and was able to send this to Codacy for analysis. Unfortunately, the percentages are still the same, meaning lines must still be counted erroneously. Still, I got a step further, so thank you so far! I will keep trying to find out what is wrong and keep this thread updated. If you have any other suggestions in the meantime, let me know! – Gijs Jun 04 '17 at 17:42
  • By the way, it does still end with the following output `-------------------------------------------------- ---------------------RESULTS---------------------- PASSED: 32 FAILED: 0 SKIPPED: 0 TOTAL: 32 -------------------------------------------------- -------------------------------------------------- spacejam: phantomjs exited with code: 0 spacejam: killing meteor spacejam: Unknown error with exit code 'null'. Exiting. ` It also does this after running the command to generate the actual coverage reports. – Gijs Jun 04 '17 at 18:10
  • It seems I've got it working now. The percentages on Codacy match the percentages in my html report. Looking at the html report more closely, it seems the coverage numbers were correct after all. It was simply the drill down that was showing odd behavior. So, the conclusion is that it worked after all, but it took Codacy's second opinion to confirm this to me. – Gijs Jun 05 '17 at 07:25
  • Yeah I should publish my own fork of spacejam directly on npm to stop this confusion.. Nice to hear that istanbul gives you the a correct coverage percentage ! Does Codacy show you the deep view without error ? – Ser Jun 05 '17 at 11:57
  • Yes, the deep view on Codacy leaves no doubt about whether the lines, functions or branches have been covered! :) – Gijs Jun 05 '17 at 13:25