Connection Pool "Leak" in development. Could it be due to JUnit test setup?

Question

I am trying to track down a database connection pool "leak" in development and I am wondering if it is resulting from how the unit tests are set up. Something is grabbing database connections from the Glassfish pool and not closing them when done. Eventually the pool max connections are used up and the application is unable to get any new db connections.

Our JUnit tests get a connection from the pool in the setUp() method and then close that connection in the tearDown() method. Can we be sure that the tearDown() method will ALWAYS run? If an unhandled exception occurs, could the tearDown() method be bypassed?

Any other ideas on what we should look for?

I should note that we are using Jakarta Cactus to run these unit tests on the Glassfish application server.

Are you sure that only the tests are using the connections and not the application ? Do you get connection leaks when the application is used without the automated tests ? — Mozart Brocchini, Mar 10 '11 at 14:46
Both the tests and the application are using db connections. I am not sure which one is to blame for the leaks. The unit tests stress the system much more than we can manually. We only see the problem when the unit tests are run. — Shane, Mar 10 '11 at 15:04
Try commenting out everything in your test except the open and close code. At least you'll know that part is working properly and it might point you towards the application being the issue. — Preston, Mar 11 '11 at 21:16

Mark Bakker · Answer 1 · 2011-03-10T15:01:25.863

1

A connection is returned to the pool when

A connection is returned to the pool when you close it pragmatically (finally block!)
A full garbage collection occurs
When the java process is killed (stop server, end of unit cycle)

In short it is highly unlikely your pooling problems are caused by running unit tests.

edited Mar 10 '11 at 15:01

answered Mar 10 '11 at 14:50

Mark Bakker

1,278
8
19

Mozart Brocchini · Accepted Answer · 2012-05-07T18:53:49.490

One suggestion to prevent and report the database connection leaks:

First find out the scope of each connection.

Example, for many web applications a connection is needed in the scope of a request.

With the scopes defined all you need to do is to close the connection in a deterministic way by doing it at the end of the scope life cycle.

One way to of asserting that a database connection is always closed in a web application is to create a servlet filter that will get the connection when a request comes in and close the connection when the response is sent. The connection can be passed from the filter to other objects by putting it in a ThreadLocal variable.

Another example of scope is when a connection is needed per transaction. You may want to use the Execute Around Method pattern to get the connection before the scope begin and close it at the end in a deterministic way.

If you implement any of these ideas you may even log which connections were not closed before you close it to help identify the leak.

Good luck, I hope this helps, please let me know otherwise.

Update:

I just solved a database connection leak in legacy code by adding debugging parameters to the database connection pool implementation apache DBCP. Even if you don't want to use DBCP in production you could still set it up in test just to detect the exact line code that borrowed the unclosed connection.

In my environment I used tomcat with the JNDI datasource config like so :

  <Resource auth="Container"
       name="jdbc/APP_NAME"
       username="user"
       password="password"
       url="jdbc:oracle:thin:@server.domain:1521:development"    
       type="javax.sql.DataSource"
       driverClassName="oracle.jdbc.driver.OracleDriver"

       maxIdle="10"
       maxWait="5000"
       maxActive="10"
       validationQuery="select 1 from dual"
       validationInterval="30000"
       testOnBorrow="true"
       testOnReturn="false"
       testWhileIdle="true"
       timeBetweenEvictionRunsMillis="5000"
       numTestsPerEvictionRun="3"
       minEvictableIdleTimeMillis="30000"


 <!-- These 3 settings saved me hours of headache -->

 logAbandoned="true" <!-- Will report the stacktrace of the faulty code --> 

 removeAbandoned="true" <!-- Will remedy the connection starvation while leaky code is not fixed-->

 removeAbandonedTimeout="60"<!-- Interval for fixing connection starvation while leaky code is not fixed-->

 />

See : Apache DBCP configuration

score 0 · Answer 3 · answered Mar 10 '11 at 14:50

I have never seen that tearDown gets missed IF it is a proper tearDown (proper method signatur or annotation depending on the JUnit version)

But: You might skip parts of the tearDown when an exception gets thrown inside the tearDown.

I'd test your theory by running your TestSuite and watching the connection pool. Sounds fairly easy to me.

Connection Pool "Leak" in development. Could it be due to JUnit test setup?

3 Answers3