4

I am working on a framework which uses some third party libraries. The frameworks clients may use the code in multi-threaded context but the framework itself does not spawn Threads or use shared memory or locks/synchronization. I am seeing some multithreading issues in the client environment and looking for a deterministic multi-threading testing framework as the problem seems to be occuring in the third party framework.

Tried so Far:

1) Able to reproduce it in a few runs in the jmeter performance testing environment

2)Able to reproduce it in a not so consistent way by stress testing using the testng multithreading annotation. (@Test(threadPoolSize = 10, invocationCount = 100, timeOut = 10000))

Cons: This just executes the method in a threadpool and does not guarantee interleaving/static code analysis or provide a way to deterministically reproduce.

3) Tried Junit's ActiveTestSuite and @ConcurrentJunitRunner. Its similar to the testng annotation.

4) MultiThreadedTC : seems to work well for lock/synchronization/shared memory based multithreading code

5) GroboUtils Seems to oriented more towards testing/monitoring shared-memory

6) IBM ConTest seems to be licensed and there is no trial version for us to ensure that it fits our needs and ask our management to purchase it.

Any suggestions similar to Microsoft Chess or HelGrind/DRD for Java which could record specific thread inter-leavings through static code analysis or byte code instrumentation and replay for debugging purposes.?

I am also looking at integrating this tool to our unit test/build process to avoid future occurences of similar error.

Yeshvanthni
  • 207
  • 2
  • 15
  • What type of issues do you see? If you experience race conditions [vmlens](http://vmlens.com) might help. – Thomas Krieger Dec 18 '13 at 20:34
  • Thanks. I just tried vmlens on my project.I believe the project is too complex for the agent to handle at this point.I already have suspicion on a particular class and field where the race condition might be happening. If you could provide an inclusion filter or test "subject" feature rather than the exclusion filter it might really be useful. – Yeshvanthni Dec 20 '13 at 02:36
  • See [Can a multi-threaded program ever be deterministic?](http://stackoverflow.com/questions/3830347/can-a-multi-threaded-program-ever-be-deterministic) – Janus Troelsen Feb 08 '15 at 14:17

1 Answers1

2

The main challenge of record&replay systems that reproduce concurrency bugs is the large amount of information that needs to be recorded. To record the thread interleaving of shared variables generates very big logs and big slowdowns. Therefore, the most recent research works attempt to perform partial logging and then use SMT solvers to complete the missing (not recorded) parts of the interleavings.

To the extent of my knowledge, the most recent advance in this area is a system called Symbiosis. Open source implementations for java/c/c++ are available (as well as tests, examples and the published paper) in this URL:

http://www.gsd.inesc-id.pt/~nmachado/software/Symbiosis_Tutorial.html

Hope it helps. Sorry for the late reply, but I joined recently

João Matos
  • 6,102
  • 5
  • 41
  • 76