How to test something in Multi-Threaded Environment using Java

Question

• How do I test something like this in multithreaded environment. I know it's gonna fail, cause this code is not thread-safe. I just wanna know how can i prove it? Creating bunch of threads and trying to add with those different threads? This code is intentionally not written properly cause of testing purposes !!!

public class Response_Unit_Manager {

private static HashMap<String, Response_Unit> Response_Unit_DB =
        new HashMap<> ();

/**
 * 
 * This subprogram adds a new Response_Unit to the data store.  The
 * new response unit must be valid Response_Unit object and it's ID must be 
 * unique (i.e., must not already exist in the data store.
 * 
 * Exceptions Thrown:  Null_Object_Exception
 */
public static void Add_Response_Unit (Response_Unit New_Unit) 
        throws Null_Object_Exception, Duplicate_Item_Exception {
    String Unit_ID = New_Unit.Unit_ID ();

    if (New_Unit == null)
        throw new Null_Object_Exception ();

    else if (Response_Unit_Exists (Unit_ID))
        throw new Duplicate_Item_Exception (Unit_ID);

    else
        Response_Unit_DB.put (Unit_ID, New_Unit);
} //end Add_Response_Unit

Answer 1

You may get lucky and see a failure when running a test, but non-failing code doesn't mean that it's thread-safe code. The only automated ways to check thread-safety is with some static analysis tools that let you put annotations on methods/classes and scan for potential issues. For example, I know FindBugs support some annotations and does concurrency checking based on them. You should be able to apply this to your single Tester class. There is still a lot of room for improvement in the industry on this topic, but here are some current examples:

• http://robertfeldt.net/publications/grahn_2010_comparing_static_analysis_tools_for_concurrency_bugs.pdf
• http://homepages.inf.ed.ac.uk/dts/students/spathoulas/spathoulas.pdf

Answer 2

As others have noted, you can't write a test that will guarantee failure as the thread schedule might "just work out", but you can write tests that have a very low probability of passing if there are thread safety issues. For example, you're code attempts to disallow duplicate items in your DB but due to thread safety issues it can't do that. So spawn a ton of threads, have them all wait on a CountdownLatch or something to maximize your chances of triggering the race, then have them all try to insert the same item. Finally you can check that (a) all but one thread saw a Duplicate_Item_Exception and (b) Response_Unit_DB contains only a single item. For these kinds of tests you can also run it several times (in the same test) to maximize your chances of triggering the issue.

Here's an example:

@Test
public void testIsThreadSafe() {
   final int NUM_ITERATIONS = 100;
   for(int i = 0; i < NUM_ITERATIONS; ++i) {
       oneIsThreaSafeTest();
   }
}

public void oneIsThreadSafeTest() {
    final int NUM_THREADS = 1000;
    final int UNIT_ID = 1;
    final Response_Unit_Manager manager = new Response_Unit_Manager();
    ExecutorService exec = Executors.newFixedThreadPool(NUM_THREADS);
    CountdownLatch allThreadsWaitOnThis = new CountdownLatch(1);
    AtomicInteger numThreadsSawException = new AtomicInteger(0);

    for (int i = 0; i < NUM_THREADS; ++i) {
        // this is a Java 8 Lambda, if using Java 7 or less you'd use a
        // class that implements Runnable
        exec.submit(() -> {
            allThreadsWaitOnThis.await();
            // making some assumptions here about how you construct
            // a Response_Unit
            Response_Unit unit = new Response_Unit(UNIT_ID);
            try {
                manager.Add_Response_Unit(unit);
            } catch (Duplicate_Item_Exception e) {
              numThreadsSawException.incrementAndGet();
            }
        });

        // release all the threads
        allThreadsWaitOnThis.countdown();

        // wait for them all to finish
        exec.shutdown();
        exec.awaitTermination(10, TimeUnits.MINUTES);

       assertThat(numThreadsSawException.get()).isEqualTo(NUM_THREADS - 1);
}  

You can construct similar tests for the other potential thread safety issues.

Answer 3

The easiest way to find errors with testing, like the one which is contained in your class, is to use a Testrunner like for example the following:

package com.anarsoft.mit;


import java.util.concurrent.atomic.AtomicInteger;


public class Test_Response_Unit_Manager  implements Runnable {


private final AtomicInteger threadCount = new AtomicInteger();

public void test() throws Exception
{
    for(int i = 0; i < 2 ;i++)
    {
        Thread thread = new Thread(this, "Thread " + i);
        this.threadCount.incrementAndGet();
        thread.start();
    }

    while( this.threadCount.get() > 0 )
    {
        Thread.sleep(1000);
    }

    Thread.sleep(10 * 1000);
}


public void run()
{
    exec();
    threadCount.decrementAndGet();

}

protected  void exec()
{
    Response_Unit_Manager.Add_Response_Unit(new Response_Unit(Thread.currentThread().getId()));
}

public static void main(String[] args) throws Exception
{
    (new Test_Response_Unit_Manager()).test();
}
}

And to use a dynamic race condition detection tool like http://vmlens.com, a lightweight race condition detector. This will show you the following race conditions:

And the stacktraces leading to the bug. On the left the write and one the right the read.

http://vmlens.com works with eclipse, so it depens on the ide you are using, if its useful for you