Is there any annotation which generates time required for each step to execute from automated test

Question

I am using TestNG to execute this test. On execution of this test I get total time required to execute this whole test. Now the requirement is I need to record time taken for each step to execute from this test. Below is my automation test. This has 3 steps.

@Test(priority = 1, description = "Login to Oasis")
public void verifyFilter(){
//Step#1
    navMenu = loginToApplication(username, password);
    Assert.assertTrue(navMenu.isDisplayed(getWebDriverInstance()), "Login Failed!!!");

//Step#2
    advancedSearchForm = navMenu.clickSearchAllTitles();
    if (!advancedSearchForm.isDisplayed(getWebDriverInstance())) {
        SimpleSearchFormElement simpleSearchForm = new SimpleSearchFormElement(getWebDriverInstance());
        advancedSearchForm = simpleSearchForm.openAdvancedSearchForm();
    }
Assert.assertTrue(advancedSearchForm.isDisplayed(getWebDriverInstance()), "Advanced Search Form is not displayed!");
//Step#3
    collectionFilterDialog = advancedSearchForm.openCollectionFilterDialog();
    boolean flag = collectionFilterDialog.isCollectionSelectableListDisplayed();
    Assert.assertTrue(flag, "Collection Filer dialog is not displayed!");
}

you can use `@QAFTestStep` annotation from [QAF](https://github.com/cbeust/testng/wiki/3rd-party-extensions#frameworks) to your method which will do the needful. You will get [detailed report](https://qmetry.github.io/qaf/latest/qaf_reporting.html) with step details, status, time and command log. — , Oct 27 '16 at 08:00

score 4 · Answer 1 · edited Oct 27 '16 at 05:59

I am using QAF for Test Automation, its inbuilt reporter generates time required for each step to execute and displays in report. I found QAF is the easiest way for detailed reporting.

with QAF your code can look like as below

@Test(priority = 1, description = "Login to Oasis")
public void verifyFilter(){
//Step#1
    CommonStep.startTransaction("Step#1: name of the step");

    navMenu = loginToApplication(username, password);
    Assert.assertTrue(navMenu.isDisplayed(getWebDriverInstance()), "Login Failed!!!");

    CommonStep.stopTransaction();

    //Step#2
    CommonStep.startTransaction("Step#2: name of the step");

    advancedSearchForm = navMenu.clickSearchAllTitles();
    if (!advancedSearchForm.isDisplayed(getWebDriverInstance())) {
        SimpleSearchFormElement simpleSearchForm = new SimpleSearchFormElement(getWebDriverInstance());
        advancedSearchForm = simpleSearchForm.openAdvancedSearchForm();
    }
    Assert.assertTrue(advancedSearchForm.isDisplayed(getWebDriverInstance()), "Advanced Search Form is not displayed!");
        CommonStep.stopTransaction();

//Step#3
    CommonStep.startTransaction("Step#3: name of the step");

    collectionFilterDialog = advancedSearchForm.openCollectionFilterDialog();
    boolean flag = collectionFilterDialog.isCollectionSelectableListDisplayed();
    Assert.assertTrue(flag, "Collection Filer dialog is not displayed!");

    CommonStep.stopTransaction();

}

above way is on the fly step declaration. Another way to define step:

@QAFTestStep(description="login using {user} and {password}")
public void login(String userName, String pwd) {
    //your code here
}

Call this method in testNG test, you will find step in report. Another benefit of teststep is you can call this step in bdd also using description.

yes, with testng and QAF implementation you can add startTransaction and stopTransation steps in code to consider it step or you can create method with @QAFTestStep and call that method in test case. — user861594, Oct 27 '16 at 05:43
I am getting error when using QAF jars. java.lang.NoClassDefFoundError: org/apache/commons/lang/StringUtils at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:763) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) at java.net.URLClassLoader.defineClass(URLClassLoader.java:467) at java.net.URLClassLoader.access$100(URLClassLoader.java:73) — user3766763, Oct 28 '16 at 18:20
I have added QAF jars into projects build path and using Step as below @QAFTestStep(description="login using {user} and {password}") public void step1(){ System.out.println("Login test"); } @Test public void test(){ step1(); } While running this test error in above comment is occured java.lang.NoClassDefFoundError: org/apache/commons/lang/StringUtils at java.lang.ClassLoader.defineClass1(Native Method) — user3766763, Oct 28 '16 at 18:22
Are you using any dependency management like ivy or maven? If so add QAF as dependency or refer [qaf dependencies](https://github.com/qmetry/qaf/blob/master/ivy.xml), download add required jars into your classpath. you also can refer [blank-project](https://github.com/qmetry/qaf-blank-project). — user861594, Nov 02 '16 at 17:17

vins · Answer 2 · 2016-10-27T00:53:26.053

1

TestNG is a unit testing framework. For unit testing, each and every method annotated with @Test is a test.

But for QA automation engineers, I would consider this as a step. that is, The class is an end-to-end test. All the test methods will be the steps for the test/workflow.

So you can design your TestNG class as shown here - your time tracking can also be done.

@Test(description = "Login to Oasis")
public void step1(){

    navMenu = loginToApplication(username, password);
    Assert.assertTrue(navMenu.isDisplayed(getWebDriverInstance()), "Login Failed!!!");

}

@Test(dependsOnMethods="step1", description = "Open Advanced Search Form")
public void step2(){

    advancedSearchForm = navMenu.clickSearchAllTitles();
    if (!advancedSearchForm.isDisplayed(getWebDriverInstance())) {
        SimpleSearchFormElement simpleSearchForm = new SimpleSearchFormElement(getWebDriverInstance());
        advancedSearchForm = simpleSearchForm.openAdvancedSearchForm();
    }
    Assert.assertTrue(advancedSearchForm.isDisplayed(getWebDriverInstance()), "Advanced Search Form is not displayed!");

}

@Test(dependsOnMethods="step2", description = "Open Collection Filter Dialog")
public void step3(){

    collectionFilterDialog = advancedSearchForm.openCollectionFilterDialog();
    boolean flag = collectionFilterDialog.isCollectionSelectableListDisplayed();
    Assert.assertTrue(flag, "Collection Filer dialog is not displayed!");

}

edited Oct 27 '16 at 00:53

answered Oct 26 '16 at 21:43

vins

15,030
3
36
47

1

I think "dependsOnMethods" is better than "priority" in that case because it highlights steps and their dependencies between them. – juherr Oct 27 '16 at 00:50
@JulienHerr, I agree. – vins Oct 27 '16 at 00:51
This is not a correct approach as one test converted in `n` tests and you will get `n` results in report rather than one for single test. Conceptually also it's not right way because you are considering Test as Step. – user861594 Oct 27 '16 at 08:32
@user861594, yes each step is a test actually! you are checking if login is successful, the window is present etc. all these are tests (also the steps of the workflow). Everyone will have their own way to design their tests. There is nothing right or wrong here. this is another approach. I wont say it is wrong. If the OP does not want to use any external lib, or some future readers, this answer might be helpful. – vins Oct 27 '16 at 13:34
This solution works well, I get the time in testNG report for each step as test, but I will loose mapping of each test in automation script to each test in test management tool. So this cannot be right choice. – user3766763 Oct 27 '16 at 17:58

score 0 · Answer 3 · answered Oct 31 '16 at 05:26

I would be curious to understand the need to find out the execution time at every action ? what does it serve. If you are still required to be doing this, I would suggest that you leverage something like below

For UI based automation tests, you can make use of EventFiringWebDriver. This Selenium implementation provides you with an ability to plugin your hooks to listen to various different events that arise out of Selenium actions. So you should be able to build a Time logger, that implements a WebDriverEventListener which you can wire in, to the EventFiringWebDriver and have it internally call Reporter.log() (Reporter is a TestNG class). You can read more about eavesdropping into webdriver events in my blog post here.

For Non UI based automation tests, you can perhaps consider one of the below two

Have your API utilities log the time individually via the Reporter.log whenever each of your methods are invoked (This would mean you would do code changes in all of your utility methods to capture start and end time to calculate the time taken).
You make use of AOP, wherein you define pointcuts via annotations and have your AOP interceptor take care of logging the timings. You can refer more about it here.

Is there any annotation which generates time required for each step to execute from automated test

3 Answers3