1

Hi I have a feature complete web-app written using AngularJS (1.5.11) and now I'm getting started with unit testing using karma (0.12.37), grunt-karma (0.8.3), karma-chrome-launcher (0.1.12), karma-phantomjs-launcher (1.0.4), phantomjs-prebuilt (2.1.14), jasmine-promise-matchers (2.3.0) and karma-jasmine (0.1.6), with a 1.3.x Jasmine version.

I'm not very confident in testing asynchronous stuff, so I started googling around and I always end up seeing the only mandatory thing to run AngularJS async tests is a $rootScope/$scope.$apply/$digest right after the async function has been called.

Eventually I found someone suggesting me to use runs() and waitsFor() and the test in this plunkr in particular runs smoothly when using the chrome-launcher but fails when using phantomjs-launcher, throwing an error like the following:

Expected { myError : { error : 'error_message' }, line : <factory's line of code which throws the error>, sourceURL : 'path/to/factory.js', stack :

     <function throwing error> B@path/to/factory.js:<factory's line of code which throws the error>
     <"async" function> A@path/to/factory.js:<factory's line of code which calls B()>
     path/to/factory-spec.js:<the following line of code: var promise = HandleService.A();>
     invoke@path/to/angular/angular.js:4771:24
     WorkFn@path/to/angular-mocks/angular-mocks.js:3130:26
     execute@path/to/node_modules/karma-jasmine/lib/jasmine.js:1145:22
     next_@path/to/node_modules/karma-jasmine/lib/jasmine.js:2177:38
     start@path/to/node_modules/karma-jasmine/lib/jasmine.js:2130:13
     execute@path/to/node_modules/karma-jasmine/lib/jasmine.js:2458:19
     next_@path/to/node_modules/karma-jasmine/lib/jasmine.js:2177:38
     start@path/to/node_modules/karma-jasmine/lib/jasmine.js:2130:13
     execute@path/to/node_modules/karma-jasmine/lib/jasmine.js:2604:19
     next_@path/to/node_modules/karma-jasmine/lib/jasmine.js:2177:38
     start@path/to/node_modules/karma-jasmine/lib/jasmine.js:2130:13
     execute@path/to/node_modules/karma-jasmine/lib/jasmine.js:2604:19
     next_@path/to/node_modules/karma-jasmine/lib/jasmine.js:2177:38
     onComplete@path/to/node_modules/karma-jasmine/lib/jasmine.js:2173:23
     finish@path/to/node_modules/karma-jasmine/lib/jasmine.js:2561:15
     path/to/node_modules/karma-jasmine/lib/jasmine.js:2605:16

     next_@path/to/node_modules/karma-jasmine/lib/jasmine.js:2187:24
     onComplete@path/to/node_modules/karma-jasmine/lib/jasmine.js:2173:23
     finish@path/to/node_modules/karma-jasmine/lib/jasmine.js:2561:15
     path/to/node_modules/karma-jasmine/lib/jasmine.js:2605:16

     next_@path/to/node_modules/karma-jasmine/lib/jasmine.js:2187:24
     onComplete@path/to/node_modules/karma-jasmine/lib/jasmine.js:2173:23
     finish@path/to/node_modules/karma-jasmine/lib/jasmine.js:2432:15
     path/to/node_modules/karma-jasmine/lib/jasmine.js:2459:16

     next_@path/to/node_modules/karma-jasmine/lib/jasmine.js:2187:24
     path/to/node_modules/karma-jasmine/lib/jasmine.js:2167:23' }, pending : undefined, processScheduled : false } } to be rejected with { myError : { error : 'error_message' } }.

So I started to think Chrome results were false positive, and I needed to rewrite the async tests: hence I tried with something like this plunkr, but now the tests fail both in PhantomJs and Chrome with the expected timeout message:

timeout: timed out after 1500 msec waiting for A should catch an error

N.B.: I can't update Jasmine to version 2.0 and use the done parameter mechanism now and if I got it right I should not even manually trigger a $rootScope.$apply/$digest when using jasmine-promise-matchers.

How can I write my async tests properly for this kind of async functions catching custom errors and async functions in general?

Gargaroz
  • 313
  • 9
  • 28
  • Offtopic: don't start with an obsolete testing framework. Use Jasmine ~2.6. Please don't reply, just take it. Ontopic: I find your question too broad / unclear: I don't see any code in the question ([the volatile](https://stackoverflow.com/help/how-to-ask) Plunkr link does not count). Even the Plunkr is not [mcve](https://stackoverflow.com/help/mcve) enough. Guide: Note that the test runner may not handle any `expect()`ations if done. You've to [use `waitsFor()` and `runs()`](https://jasmine.github.io/1.3/introduction.html#section-Asynchronous_Support). Test for the promise state or set a flag – try-catch-finally Jul 12 '17 at 17:55
  • Offtopic: ... Ontopic: the truth is I don't know how to set up something resembling my testing environment (that's why I linked every information about the grunt tasks involved) in a plunkr, but as I said [this gentleman already told me I should use waitsFor() and runs(), I got it](https://stackoverflow.com/questions/33467485/how-to-test-that-an-angular-factory-function-using-a-promise-is-throwing-an-erro/33499275?noredirect=1#comment77031514_33499275), hence I provided a plunkr with my "best effort". – Gargaroz Jul 13 '17 at 08:24
  • 1
    The gentleman was wrong and wasn't confident about how the things work. You will rarely ever need `done` in AngularJS tests - and even more so, runs/waitsFor that smell even worse. – Estus Flask Jul 17 '17 at 23:48
  • @estus Interesting, so are you telling me, good sir, I shall not use runs() and waitsFor() in my AngularJS async tests? So how shall I write the one I wrote about in a way it doesn't generate a false positive or whatsoever? – Gargaroz Jul 18 '17 at 08:10
  • @estus, Using a library which hides async promise behavior does not actually prove that I am wrong, it is simply another way to solve a problem. But I agree that it is a way more cleaner solution, thanks for the lib suggestion. – Michael Radionov Jul 18 '17 at 18:08
  • 1
    @MichaelRadionov The point here is that $q promises do not have async behaviour per se. They are synchronous (as long as the code that they promise is), this and their connection with scope digests really differ them from other promise implementations. From what I know of Angular internals, $q and other built-in services were built like that exactly to make Jasmine testing cleaner and faster. – Estus Flask Jul 18 '17 at 18:15
  • @estus, Wow, I did not actually know about that, thanks for the information. I'll figure it out and update the reference post to fix myself and spread the word. – Michael Radionov Jul 18 '17 at 18:57
  • @MichaelRadionov You're welcome. This is not the most obvious part of Angular 1 design, I'm not even sure if it is mentioned in the docs. It was my downvote on your answer because I found it misleading in that context. The vote is locked until there will be changes in the post. Please, notify me when you'll fix it so I could change it to upvote. – Estus Flask Jul 18 '17 at 19:14

1 Answers1

2

Angular 'async' tests are generally synchronous, thus waitsFor and runs are unnecessary and apparently harmful.

Indeed, jasmine-promise-matchers don't need to trigger a digest manually to execute $q promises since this is done internally.

The problem here is race condition. First runs seems to run after $rootScope.$digest(), and catch block is never executed - so is second runs.

Instead, it should be tested synchronously:

  it('it actually throws an error, yay', function () {
    var promise = HandleService.A();

    expect(promise).toBePromise();
    expect(promise).toBeRejectedWith(jasmine.objectContaining({
      myError: {error: 'error_message'}
    }));
  });
Estus Flask
  • 206,104
  • 70
  • 425
  • 565
  • Your solution sounds reasonable to me, alas, the test you wrote passes (if it's not a false positive) when using Chrome as testing browser, while fails when using PhantomJS with the very error I wrote about in my question: `Expected { myError : { error : 'error_message' }, line : , sourceURL : 'path/to/factory.js', stack : etc.`. Can you help me to find what's wrong with this? – Gargaroz Jul 18 '17 at 11:05
  • 1
    It's hard to say. It depends on what is the actual code you're testing. The code you've posted contains nothing that would make it to fail like that. Generally you pack Phantom with polyfills (core-js or else) and it works like intended. If you didn't, that's where you start. Please, provide what Phantom version you're using,full error (not mangled) and your real code if it differs. – Estus Flask Jul 18 '17 at 11:47
  • Ok check my edited question out, I did my best to provide as many informations as I can about involved packages version and error stack, let me know if you need to know anything else. – Gargaroz Jul 18 '17 at 13:08
  • I see. Full error message actually helps, it is important part that should never be truncated. Still not sure why it's ok for Chrome but not ok for Phantom, Probably a bug in promise matchers, or your real code differs from the one you've posted. – Estus Flask Jul 18 '17 at 13:29
  • 1
    This likely can be fixed with `expect(promise).toBeRejectedWith(jasmine.objectContaining({myError: {error: 'error_message'}}))`, [like the documentation shows](https://github.com/bvaughn/jasmine-promise-matchers#toberejectedwith) – Estus Flask Jul 18 '17 at 13:32
  • It's super effective! How comes I have to do that `jasmine.objectContaining` wrapping? – Gargaroz Jul 18 '17 at 13:39
  • 1
    Error message shows that error object has extra props, like if a promise was rejected with `Error` and not plain object. This can be handled with `objectContaining` partial match. – Estus Flask Jul 18 '17 at 14:11
  • Thank you very much for the plain simple explanation, one last question before StackOverflow starts complaining too much about the number of comments to this answer: is there **any side effect** to be aware of while using the `objectContaining` partial matcher? – Gargaroz Jul 18 '17 at 14:22
  • 1
    SO warnings can be efficiently ignored. The side effect is obviously that it matches the object that has extra props that play some role in app logic but weren't taken into account. Generally the stricter tests are, the better. In current case there are no caveats, except the fact that it is unclear what's going on and how comes that it's not a plain object. – Estus Flask Jul 18 '17 at 14:29
  • Thank you very much for all your effort in answering this question, I was wondering, should we add that `jasmine.objectContaining` thing to your answer? – Gargaroz Jul 18 '17 at 15:11