Setting testfailure in testng gives positive return

Question

I want to set the test result of a method in testng to failure if there is an error. I found that this line of code should do the trick

Reporter.getCurrentTestResult().setStatus(ITestResult.FAILURE);

But if I provoke an error I will still get no failure returned from my total test run

===============================================
LoginTesten
Total tests run: 4, Failures: 0, Skips: 0
===============================================

In the above run there should be 2 failures, but there is nothing. The rule to set the test result is in a different class then where the test is run from. It is set in a class where I put al my checkpage code so I that I have no bombastic test.

The code where it is set

public void checkTextOnPage(String text, boolean expected,String errorMessage, ArrayList<WebDriver> browsers, Logger Logger, String map) throws IOException {
        for(WebDriver driver: browsers){
            try {
                Assert.assertEquals(driver.getPageSource().contains(text), expected, errorMessage);
            }catch (AssertionError e){}
            Capabilities dataBrowser = ((RemoteWebDriver)driver).getCapabilities();
            if(driver.getPageSource().contains(text) == expected){
                Logger.info("[" + this.dateFormat.format(this.date) + "]" +" Browser:" + dataBrowser.getBrowserName() + " -- Version:" + dataBrowser.getVersion() + " -- Text : " + text + " => PRESENT");
            }else if (driver.getPageSource().contains(text) != expected){
                Logger.warning("[" + this.dateFormat.format(this.date) + "]" +" Browser:" + dataBrowser.getBrowserName() + " -- Version:" + dataBrowser.getVersion() + " -- Text : " + text + " => NOT PRESENT");
                File screenshot = ((TakesScreenshot)driver).getScreenshotAs(OutputType.FILE);
                FileUtils.copyFile(screenshot, new File(map + "text_" + text + "_NotPresent_" + this.screenshotCount + "_" + dataBrowser.getBrowserName() + ".png"));
                this.screenshotCount++;
                Reporter.getCurrentTestResult().setStatus(ITestResult.FAILURE);
            }
        }
    }

this is my test that is being run

public void loginIMWWelcomeNOT() throws InterruptedException, IOException {
    this.getLogin().goToPage(this.getDriverList(), TestConstants.Test_URL);
    this.getBrowserMovement().typeTextElementXpath(".//*[@name='username']", this.getDriverList(), "intix1");
    this.getBrowserMovement().typeTextElementXpath(".//*[@name='password']", this.getDriverList(), "Jasmine11");
    this.getBrowserMovement().clickElementXpath(".//*[@name='logon']", this.getDriverList());
    this.getCheckPage().checkTextOnPage("grazjfnzeofnez", true, "We komen niet terecht op de welkom pagina", this.getDriverList(), this.getLOGGER(), this.getScreenshotMap());
    System.out.println(Reporter.getCurrentTestResult());
}

The output I get from that code is

apr 04, 2016 4:50:37 PM Data.CheckPage checkTextOnPage
WARNING: [2016-04-04 165030] Browser:chrome -- Version:49.0.2623.110 -- Text : grazjfnzeofnez => NOT PRESENT
[TestResult name="" status=FAILURE method=LoginTesten.loginIMWWelcomeNOT()[pri:0, instance:Data.Tests.LoginTesten@1e643faf] output={null}]

So it is being set but it's not picked up by anything. Is it the problem I set it in the other class or something else?

score 0 · Answer 1 · edited May 23 '17 at 12:07

0

Your tests should not directly interact with your reporting. To indicate an error in JUnit, simply throw an exception or asset in your tests.

edited May 23 '17 at 12:07

Community

1
1

answered Apr 04 '16 at 15:13

Jan Nielsen

10,892
14
65
119

But if I count on the assert it will just drop out of the test without any logging. I could catch it and add the logging but it's the same result the test will completly fail and I want to run it until the end – stevedc Apr 05 '16 at 07:13
That the reponsibility of your build system, e.g., http://stackoverflow.com/questions/4174696/making-maven-run-all-tests-even-when-some-fail. – Jan Nielsen Apr 05 '16 at 16:44

score 0 · Accepted Answer · answered Apr 07 '16 at 03:17

@stevedc Catch the error and then at the end of your test output it in the Assert failure as the error message. Something like

public void checkTextOnPage(String text, boolean expected,String errorMessage, ArrayList<WebDriver> browsers, Logger Logger, String map) throws IOException {
    for(WebDriver driver: browsers){
        try {
            Assert.assertEquals(driver.getPageSource().contains(text), expected, errorMessage);
        }catch (AssertionError e){}
        Capabilities dataBrowser = ((RemoteWebDriver)driver).getCapabilities();
        if(driver.getPageSource().contains(text) == expected){
            Logger.info("[" + this.dateFormat.format(this.date) + "]" +" Browser:" + dataBrowser.getBrowserName() + " -- Version:" + dataBrowser.getVersion() + " -- Text : " + text + " => PRESENT");
        }else if (driver.getPageSource().contains(text) != expected){
            Logger.warning("[" + this.dateFormat.format(this.date) + "]" +" Browser:" + dataBrowser.getBrowserName() + " -- Version:" + dataBrowser.getVersion() + " -- Text : " + text + " => NOT PRESENT");
            File screenshot = ((TakesScreenshot)driver).getScreenshotAs(OutputType.FILE);
            FileUtils.copyFile(screenshot, new File(map + "text_" + text + "_NotPresent_" + this.screenshotCount + "_" + dataBrowser.getBrowserName() + ".png"));
            this.screenshotCount++;
            Assert.assertEquals(true, false, e);
        }
    }
}

Setting testfailure in testng gives positive return

2 Answers2