How to have protractor reliable results?

Question

I'm using Protractor (v 1.3.1) to run E2E tests for my Angular 1.2.26 application.

But sometimes, tests are ok, sometimes not. It seems that sometimes the check is done before display is updated (or something like "synchronisation" problem). I try many options :

• add browser.driver.sleep instructions,
• disable effects with browser.executeScript('$.fx.off = true')
• add browser.waitForAngular() instructions

without success.

What are the bests practice to have reliables E2E tests with protractor?

JM.

Answer 1

Every time I have similar issues, I'm using browser.wait() with "Expected Conditions" (introduced in protractor 1.7). There is a set of built-in expected conditions that is usually enough, but you can easily define your custom expected conditions.

For example, waiting for an element to become visible:

var EC = protractor.ExpectedConditions;
var e = element(by.id('xyz'));

browser.wait(EC.visibilityOf(e), 10000);
expect(e.isDisplayed()).toBeTruthy();

Few notes:

• you can specify a custom error message in case the conditions would not be met and a timeout error would be thrown, see Custom message on wait timeout error:

  browser.wait(EC.visibilityOf(e), 10000, "Element 'xyz' has not become visible");
  

• you can set EC to be a globally available variable pointing to protractor.ExpectedConditions. Add this line to the onPrepare() in your config:

  onPrepare: function () {
      global.EC = protractor.ExpectedConditions;
  }
  

• as an example of a custom expected condition, see this answer

Answer 2

Another point which is very important in testing with Protractor is understanding the ControlFlow. You may find explaination and code example here : When should we use .then with Protractor Promise?

Jean-marc

Answer 3

There are two things to consider.

The first is that you should properly sequence all protractor actions (as also hinted by @jmcollin92). For this, I typically use .then on every step.

The second important thing is to make sure that a new test it(...) only starts after the previous it(...) has completed.

If you use the latest version of Protractor, you can use Jasmine 2.x and its support for signalling the completion of a test:

it('should do something', function(done) {
   clicksomething().then(function() {
     expect(...);
     done();
   });
});

Here the done argument is invoked to signal that the test is ready. Without this, Protractor will schedule the clicksomething command, and then immediately move on with the next test, returning to the present test only once clicksomething has completed.

Since typically both tests inspect and possibly modify the same browser/page, your tests become unpredictable if you let them happen concurrently (one test clicks to the next page, while another is still inspecting the previous page).

If you use an earlier version of Protractor (1.3 as you indicate), the Jasmine 1.3 runs and waitsFor functions can be used to simulate this behavior.

Note that the whole point of using Protractor is that Protractor is supposed to know when Angular is finished. So in principle, there should be no need to ever call waitForAngular (my own test suite with dozens of scenarios does not include a single wait/waitForAngular). The better your application-under-test adheres to Angular's design principles, the fewer WaitForAngular's you should need.

Answer 4

I would add that disabling ngAnimate may not be enough. You may also have to disable all transition animation by injecting CSS (What is the cleanest way to disable CSS transition effects temporarily?).