Concurrent JUnit Tests with Parameters

Question

So I'm attempting to run parallel parameterized tests. I have a solution where the same test can run in parallel with the parameters supplied for example say I have the following:

@Test
public void someTest1(){
}

@Test
public void someTest2(){
}

I can get someTest1() to run with all the parameters concurrently, but someTest2() will have still have to wait for someTest1() to complete with all parameters before executing. I was wondering if anyone knew of a solution to be able to run someTest1() with all parameters and someTest2() with all parameters concurrently? I've tried tempus-fugit concurrent test runner, which works great for tests that are not parameterized...

Below is code that I have for currently running each parameter test in parallel.

import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;

import org.junit.runners.Parameterized;
import org.junit.runners.model.RunnerScheduler;

/**
 * Class used from the following source:
 * http://jankesterblog.blogspot.com/2011/10
 * /junit4-running-parallel-junit-classes.html
 * 
 * @author Jan Kester
 * 
 */
public class Parallelized extends Parameterized {

    private static class ThreadPoolScheduler implements RunnerScheduler {
        private ExecutorService executor;

        public ThreadPoolScheduler() {
            String threads = System.getProperty("junit.parallel.threads", "16");
            int numThreads = Integer.parseInt(threads);
            executor = Executors.newFixedThreadPool(numThreads);
        }

        public void finished() {
            executor.shutdown();
            try {
                executor.awaitTermination(12, TimeUnit.HOURS);
            } catch (InterruptedException exc) {
                throw new RuntimeException(exc);
            }
        }

        public void schedule(Runnable childStatement) {
            executor.submit(childStatement);
        }
    }

    /**
     * Instantiates a new parallelized.
     * 
     * @param klass
     *            the klass
     * @throws Throwable
     *             the throwable
     */
    public Parallelized(Class<?> klass) throws Throwable {
        super(klass);
        setScheduler(new ThreadPoolScheduler());
    }
}

The code below is an example test, BaseSuite doesn't contain anything of much importance. These are being used with selenium so it just sets the webDriver. The getAllButOpera() method returns a collection of browser types that contain Internet Explorer, Firefox, and Chrome. These parameters are used to run the same test on firefox, ie, and chrome concurrently. I would like to run the two tests in the class at the same time which is what I am having trouble with.

import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertTrue;

import java.util.Collection;

import org.junit.Before;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.Parameterized.Parameters;
import org.openqa.selenium.WebDriver;


/**
 * The Class SampleSuite1.
 * 
 * @author Reid McPherson
 */
@RunWith(Parallelized.class)
public class SampleSuite1 {
    WebDriver driver;
    /**
     * Data.
     * 
     * @return the collection
     */
    @Parameters
    public static Collection<Object[]> data(){
          List<Object[]> browsers = new ArrayList<Object[]>();
    browsers.add(new String[]{"Firefox"});
    browsers.add(new String[]{"Chrome"});
    browsers.add(new String[]{"IE"});
    return browsers;
    }

    /**
     * Instantiates a new sample suite1.
     * 
     * @param type
     *            the type
     */
    public SampleSuite1(String type){
        switch (type) {
    case "FIREFOX":
        driver = new FirefoxDriver();
        break;
    case "IE":
        driver = new InternetExplorerDriver();
        break;
    case "CHROME":
        System.setProperty("webdriver.chrome.driver", PATHTOCHROMEEXE);
        driver = new ChromeDriver();
        break;
    case "OPERA":
        driver = new OperaDriver();
        break;
    default:
        throw new RuntimeException("Browser type unsupported");
    }
    // Set the timeout.
    driver.manage().timeouts().implicitlyWait(60, TimeUnit.SECONDS);
    }

    /**
     * Sets the up.
     */
    @Before
    public void setUp() {
        driver.get("http://www.google.com");
    }

    /**
     * Test navigation succeeded.
     */
    @Test
    @TestDescription("Navigation Test")
    public void navigationShouldSucceed() {
        String pageSource = driver.getPageSource();
        assertTrue(pageSource.contains("Google"));
    }

    /**
     * Test title.
     */
    @Test
    @TestDescription("This method tests the web page title.")
    public void titleShouldBeGoogle() {
        assertEquals(driver.getTitle(), "Google");
    }

    @After
    public void finished(){
    driver.close();
    }


}

Answer 1

As I told the question is in implementation of JUnit.

You can see that:

Parallelized extends Parametrized extends Suite extends ParentRunner

on the other hand:

ConcurrentTestRunner extends BlockJUnit4ClassRunner extends ParentRunner

hence they are from different hierarchies of inheritance.

And now what you have to look at is implementation of the:

org.junit.runners.ParentRunner#getChildren

method. For the org.junit.runners.BlockJUnit4ClassRunner it is:

protected List<FrameworkMethod> computeTestMethods() {
    return getTestClass().getAnnotatedMethods(Test.class);
}

which generates all methods with annotations. But for org.junit.runners.Parameterized it is:

for (int i= 0; i < parametersList.size(); i++)
  runners.add(newtestClassRunnerForParameters(getTestClass().getJavaClass(),
                parametersList, i));

And the last one gives only classes.

Proposal: override your Parallelized class with definition of org.junit.runners.ParentRunner#getChildren from BlockJUnit4ClassRunner.

Answer 2

Thanks for the help, I ended up using code from here in addition to running concurrent suites and it gave me the ability to run the tests simultaneously, but it will not run the same test at the same time.

Answer 3

I am also using the code from Jeeunit... however even the version 1.0 has a bug.

In the ConcurrentRunnerScheduler the finished method must be implemented.

So I just pulled the code and implemented it like the suiteFinished() method:

@Override
    public void finished() {
        try {
        while (!tasks.isEmpty())
            tasks.remove(completionService.take());
    }
    catch (InterruptedException e) {
        System.out.println("suite fin");
        Thread.currentThread().interrupt();
    }
    finally {
        while (!tasks.isEmpty())
            tasks.poll().cancel(true);
        executorService.shutdownNow();
    }

}

Answer 4

This is what I have tried, and it works really well for me.

public class ParallelTests {
   static ExecutorService eService ;
   public static void main(String[] args) {
      testA() ;
      testB() ;
      testC() ;
   }
   public static void testA() {
      eService = Executors.newCachedThreadPool() ;
      for (int i = 0 ; i < 10 ; i++) {
         TestA testA = new TestA() ;
         eService.execute(testA) ;
      }
      eService.shutdown() ;
      while(!eService.isShutDown()) {
      }
   }
//same for testB and testC
}

public class TestA implements Runnable {
    public TestA() {
    }
    @Test
    public myTest throws Throwable {
    }
}