HikariCP connection pool not shutting down

Question

Problem description

I am running Unit Tests in a Play Framework application. After a couple tests, they start failing and the reason why is that the PostgreSQL limit of 100 open connections is reached. Each Unit Test has a connection pool of 10, so after 10 Unit tests, the fails start. Keep in mind, the tests run very fast and the pool is created and shut down several times in less than a second. Using Debug Logging, I don't see any unexpected behavior (or I haven't enabled all of the debug informations).

Managing the connection pool

I am not managing the connection pool myself (it's managed by the Play Framework) and I can see in the log that the pool is shut down after each Unit Test.

[info] application - Creating Pool for datasource 'default'
[info] p.a.d.HikariCPConnectionPool - datasource [default] bound to JNDI as DefaultDS
[info] application - ApplicationTimer demo: Starting application at 2017-04-20T08:50:46.096Z
[info] application - ApplicationTimer demo: Stopping application at 2017-04-20T08:50:46.129Z after 1s.
[info] application - Shutting down connection pool.

In the graph of the database you can see how the pool starts the connections after every unit test and suddenly after the fails start, the open connections go down instantanly. I don't see any connection shutting down between the tests.

Example Unit Test

Because I couldn't find any good examples of Unit Tests in the style I am doing, I can not confirm that my Unit Tests are flawless (in the sense of: I am using the provided tools of Play Framework correctly). Maybe you can spot some misusage of the framework itself that would prevent the Connection Pool to be shutdown.

public class UserPersistenceTesting {
    @Inject
    private JPAApi jpaApi;

    @Before
    public void setup() {
        GuiceApplicationBuilder builder = new GuiceApplicationLoader()
                .builder(new ApplicationLoader.Context(Environment.simple()));
        Guice.createInjector(builder.applicationModule()).injectMembers(this);
    }

    @After
    public void teardown() {
        jpaApi.shutdown();
    }

    @Test
    public void testRegisterNewUser() {
        this.jpaApi.withTransaction(() -> {
            UsersRepository repository = new UsersRepositoryJPA(jpaApi);
            Date creationDate = new Date();
            User expectedUser = new User(null, "max.muster@hsr.ch", "Muster", "Max", creationDate, Gender.MALE);
            repository.registerUser(expectedUser);

            User actualUser = repository.getUserById(expectedUser.getId());

            assertEquals(expectedUser, actualUser);
        });
    }
}

What I think is the problem

I have the feeling that there is a delay between the "shutdown" of the pool and the releasing of the connections. Because the Unit Tests run so fast, the pool would not be able to be released before the Database starts throwing exceptions.

What I have ruled out

Here is a list of errors that are ruled out:

More than 10 concurrent tests run at the same time and fill the connection pool. -> I have disabled concurrent tests and the errors still occurr.

Occurring exceptions

[error] o.p.Driver - Connection error:
org.postgresql.util.PSQLException: FATAL: tut mir leid, schon zu viele Verbindungen
  Ort: Datei: proc.c, Routine: InitProcess, Zeile: 340.
  Server SQLState: 53300
        at org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:438)
        at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:222)
        at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
        at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:194)
        at org.postgresql.Driver.makeConnection(Driver.java:431)
        at org.postgresql.Driver.connect(Driver.java:247)
        at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:95)
        at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:101)
        at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:341)
        at com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:506)
[error] Test integration.database.EventPersictenceTesting.testRegisterNewEvent failed: play.api.Configuration$$anon$1: Configuration error[Cannot connect to database [default]], took 0.094 sec
[error]     at play.api.Configuration$.configError(Configuration.scala:155)
[error]     at play.api.Configuration.reportError(Configuration.scala:808)
[error]     at play.api.db.DefaultDBApi$$anonfun$connect$1.apply(DefaultDBApi.scala:48)
[error]     at play.api.db.DefaultDBApi$$anonfun$connect$1.apply(DefaultDBApi.scala:42)
[error]     at scala.collection.immutable.List.foreach(List.scala:381)
[error]     at play.api.db.DefaultDBApi.connect(DefaultDBApi.scala:42)
[error]     at play.api.db.DBApiProvider.get$lzycompute(DBModule.scala:72)
[error]     at play.api.db.DBApiProvider.get(DBModule.scala:62)
[error]     at play.api.db.DBApiProvider.get(DBModule.scala:58)
[error]     at com.google.inject.internal.ProviderInternalFactory.provision(ProviderInternalFactory.java:81)
[error]     at com.google.inject.internal.BoundProviderFactory.provision(BoundProviderFactory.java:72)
[error]     at com.google.inject.internal.ProviderInternalFactory.circularGet(ProviderInternalFactory.java:61)
[error]     at com.google.inject.internal.BoundProviderFactory.get(BoundProviderFactory.java:62)
[error]     at com.google.inject.internal.SingleParameterInjector.inject(SingleParameterInjector.java:38)
[error]     at com.google.inject.internal.SingleParameterInjector.getAll(SingleParameterInjector.java:62)
[error]     at com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:104)
[error]     at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:85)
[error]     at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:267)
[error]     at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46)
[error]     at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1103)
[error]     at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
[error]     at com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:145)
[error]     at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:41)
[error]     at com.google.inject.internal.FactoryProxy.get(FactoryProxy.java:56)
[error]     at com.google.inject.internal.SingleFieldInjector.inject(SingleFieldInjector.java:54)
[error]     at com.google.inject.internal.MembersInjectorImpl.injectMembers(MembersInjectorImpl.java:132)
[error]     at com.google.inject.internal.MembersInjectorImpl$1.call(MembersInjectorImpl.java:93)
[error]     at com.google.inject.internal.MembersInjectorImpl$1.call(MembersInjectorImpl.java:80)
[error]     at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1103)
[error]     at com.google.inject.internal.MembersInjectorImpl.injectAndNotify(MembersInjectorImpl.java:80)
[error]     at com.google.inject.internal.MembersInjectorImpl.injectMembers(MembersInjectorImpl.java:62)
[error]     at com.google.inject.internal.InjectorImpl.injectMembers(InjectorImpl.java:984)
[error]     at com.google.inject.util.Providers$GuicifiedProviderWithDependencies.initialize(Providers.java:149)
[error]     at com.google.inject.util.Providers$GuicifiedProviderWithDependencies$$FastClassByGuice$$2a7177aa.invoke(<generated>)
[error]     at com.google.inject.internal.cglib.reflect.$FastMethod.invoke(FastMethod.java:53)
[error]     at com.google.inject.internal.SingleMethodInjector$1.invoke(SingleMethodInjector.java:57)
[error]     at com.google.inject.internal.SingleMethodInjector.inject(SingleMethodInjector.java:91)
[error]     at com.google.inject.internal.MembersInjectorImpl.injectMembers(MembersInjectorImpl.java:132)
[error]     at com.google.inject.internal.MembersInjectorImpl$1.call(MembersInjectorImpl.java:93)
[error]     at com.google.inject.internal.MembersInjectorImpl$1.call(MembersInjectorImpl.java:80)
[error]     at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092)
[error]     at com.google.inject.internal.MembersInjectorImpl.injectAndNotify(MembersInjectorImpl.java:80)
[error]     at com.google.inject.internal.Initializer$InjectableReference.get(Initializer.java:174)
[error]     at com.google.inject.internal.Initializer.injectAll(Initializer.java:108)
[error]     at com.google.inject.internal.InternalInjectorCreator.injectDynamically(InternalInjectorCreator.java:174)
[error]     at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:110)
[error]     at com.google.inject.Guice.createInjector(Guice.java:96)
[error]     at com.google.inject.Guice.createInjector(Guice.java:84)
[error]     at play.api.inject.guice.GuiceBuilder.injector(GuiceInjectorBuilder.scala:181)
[error]     at play.api.inject.guice.GuiceApplicationBuilder.build(GuiceApplicationBuilder.scala:123)
[error]     at play.api.test.FakeApplication.<init>(Fakes.scala:209)
[error]     at play.test.FakeApplication.<init>(FakeApplication.java:51)
[error]     at play.test.Helpers.fakeApplication(Helpers.java:124)
[error]     at play.test.WithApplication.provideFakeApplication(WithApplication.java:46)
[error]     at play.test.WithApplication.provideApplication(WithApplication.java:33)
[error]     at play.test.WithApplication.startPlay(WithApplication.java:51)
[error]     ...
[error] Caused by: Configuration error: Configuration error[Failed to initialize pool: FATAL: tut mir leid, schon zu viele Verbindungen
[error]   Ort: Datei: proc.c, Routine: InitProcess, Zeile: 340.
[error]   Server SQLState: 53300]
[error]     at play.api.Configuration$.configError(Configuration.scala:155)
[error]     at play.api.PlayConfig.reportError(Configuration.scala:999)
[error]     at play.api.db.HikariCPConnectionPool.create(HikariCPModule.scala:70)
[error]     at play.api.db.PooledDatabase.createDataSource(Databases.scala:199)
[error]     at play.api.db.DefaultDatabase.dataSource$lzycompute(Databases.scala:123)
[error]     at play.api.db.DefaultDatabase.dataSource(Databases.scala:121)
[error]     at play.api.db.DefaultDatabase.getConnection(Databases.scala:142)
[error]     at play.api.db.DefaultDatabase.getConnection(Databases.scala:138)
[error]     at play.api.db.DefaultDBApi$$anonfun$connect$1.apply(DefaultDBApi.scala:44)
[error]     ... 88 more
[error] Caused by: com.zaxxer.hikari.pool.HikariPool$PoolInitializationException: Failed to initialize pool: FATAL: tut mir leid, schon zu viele Verbindungen
[error]   Ort: Datei: proc.c, Routine: InitProcess, Zeile: 340.
[error]   Server SQLState: 53300
[error]     at com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:512)
[error]     at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:105)
[error]     at com.zaxxer.hikari.HikariDataSource.<init>(HikariDataSource.java:71)
[error]     at play.api.db.HikariCPConnectionPool$$anonfun$1.apply(HikariCPModule.scala:58)
[error]     at play.api.db.HikariCPConnectionPool$$anonfun$1.apply(HikariCPModule.scala:54)
[error]     at scala.util.Try$.apply(Try.scala:192)
[error]     at play.api.db.HikariCPConnectionPool.create(HikariCPModule.scala:54)
[error]     ... 94 more
[error] Caused by: org.postgresql.util.PSQLException: FATAL: tut mir leid, schon zu viele Verbindungen
[error]   Ort: Datei: proc.c, Routine: InitProcess, Zeile: 340.
[error]   Server SQLState: 53300
[error]     at org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:438)
[error]     at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:222)
[error]     at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
[error]     at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:194)
[error]     at org.postgresql.Driver.makeConnection(Driver.java:431)
[error]     at org.postgresql.Driver.connect(Driver.java:247)
[error]     at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:95)
[error]     at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:101)
[error]     at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:341)
[error]     at com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:506)
[error]     ... 100 more
[error] Failed: Total 18, Failed 6, Errors 0, Passed 12
[error] Failed tests:
[error]         integration.database.EventPersictenceTesting
[error]         integration.rest.IntegrationTest
[error] (test:test) sbt.TestsFailedException: Tests unsuccessful
[error] Total time: 31 s, completed 20.04.2017 10:50:52

After running lots of tests, I have managed to get into a state where I can't log into the Database as Superuser anymore because there are connections open. The Play Application is closed. — El Mac, Apr 20 '17 at 10:55
Have you read [this similar question](http://stackoverflow.com/questions/42368523/play-tests-with-database-too-many-connections)? — brettw, Apr 20 '17 at 14:12

score 1 · Answer 1 · answered May 30 '18 at 13:59

This is an old question and I want to answer it with what I remember was the solution (or at least it is working like this now).

When I watch the code in the original question, I was not using the Play Test framework correctly. My class now extends WithApplication. I don't inject the JPAApi into the test class anymore, but instead I get it from inside the test methods. I dont need the setup() method anymore. The refactored class looks something like this:

public class UserPersistenceTesting extends WithApplication {

@Test
public void testRegisterNewUser() {
    final JPAApi jpaApi = this.app.injector().instanceOf(JPAApi.class);
    jpaApi.withTransaction(() -> {
        final UsersRepositoryJPA repository = new UsersRepositoryJPA(jpaApi);
        final Date creationDate = new Date();
        final User expectedUser = new User(null, "max.muster@hsr.ch", "Muster", "Max", creationDate, Gender.MALE);
        repository.registerUser(expectedUser);

        final User actualUser = repository.getUserById(expectedUser.getId());

        assertEquals(expectedUser, actualUser);
    });
}