0

Hi stackoverflow community,

I have a really annoying issue with my jenkins and hikaricp.

To put the issue in perspective: I have a spring boot application which uses the hikariDataSource. It works fine while developing and all test run local without any issues. But if the same project runs in jenkins as a ci job, which does exactly the same thing as I would do on my development machine it get stuck for a while in cleaning up the connection pool:

2015-08-28 15:51:52.090 DEBUG 8234 --- [l HikariPool-0)] com.zaxxer.hikari.pool.HikariPool : Before cleanup pool HikariPool-0 stats (total=10, active=0, idle=10, waiting=0) 2015-08-28 15:51:52.091 DEBUG 8234 --- [l HikariPool-0)] com.zaxxer.hikari.pool.HikariPool : After cleanup pool HikariPool-0 stats (total=10, active=0, idle=10, waiting=0) 2015-08-28 15:52:22.090 DEBUG 8234 --- [l HikariPool-0)] com.zaxxer.hikari.pool.HikariPool : Before cleanup pool HikariPool-0 stats (total=10, active=0, idle=10, waiting=0) 2015-08-28 15:52:22.091 DEBUG 8234 --- [l HikariPool-0)] com.zaxxer.hikari.pool.HikariPool : After cleanup pool HikariPool-0 stats (total=10, active=0, idle=10, waiting=0)

This is what I got for a couple of minutes and then the test continues.

In the test environment I use a h2 in-memory database.

After a couple of hours search on google and github I'm not a bit closer to a solution.

I use Spring Boot 1.2.5, HikariCP 2.4.1, Hibernate 4.3.10.final.

The configuration of the spring boot project is based on jhipster's configuration.

Many many thanks in advanced for you help and further information to solve the issue.

conscience
  • 463
  • 6
  • 21
  • Are you sure that your test harness is shutting down the HikariDataSource? If you are using spring configuration, there should be a "destroy" property somewhere wired to HikariDataSource.close() method. Maybe something like [this](http://stackoverflow.com/questions/30544063/gracefull-shutdown-for-spring-boot-application)? – brettw Aug 28 '15 at 14:24
  • Thanks for your answer. I'm not sure what you mean. My DataSource config does call the close method on shutdown. Did you mean that it is called to early? – conscience Aug 28 '15 at 16:11

2 Answers2

0

I thought the you said the pool was hanging your test. But reading it again, I don't think that is what's happening. HikariCP is going to log those cleanup stats every 30 seconds as long as it is running. So, if another thread hangs the tests, you would expect to see these continuing to log.

The presence of those logs is no indication that HikariCP is hung. From all indications in that log, there are no connections in use at all.

If you can somehow get a thread dump while the tests appear to be hung, you can see what what the threads are doing and probably catch the true culprit.

brettw
  • 10,664
  • 2
  • 42
  • 59
0

thanks to @brettw I could solve the issue. The logs did indeed lead me in the wrong direction, to think that hikaricp did something wrong.

The root cause of our long running builds was the entropy source, which is used by the embedded tomcat to build a SecureRandom instance.

According to this post: Spring Boot Actuator application won't start on Ubuntu VPS we tried to set the source to /dev/./urandom, which also doesn't work.

In the end we installed haveged, as @starmer suggested. Now it works just fine. The builds a insanly fast.

Thank you very much @brettw to point me in the right direction!

Community
  • 1
  • 1
conscience
  • 463
  • 6
  • 21