2

I have a heimdallr application that is deployed via wildfly. In this application I am using the cache functionality that is provided by the namespace immutant.caching.

The first time I deploy the application everything goes fine without error. If I attempt to redeploy the application I get the following error:

2015-05-27 18:20:54,703 ERROR [org.jboss.msc.service.fail] (ServerService Thread Pool -- 82) MSC000001: Failed to start service jboss.infinispan.web.dist: org.jboss.msc.service.StartException in service jboss.infinispan.web.dist: java.lang.IllegalStateException: Cache container has been stopped and cannot be reused. Recreate the cache container.
        at org.jboss.as.clustering.msc.AsynchronousService$1.run(AsynchronousService.java:91)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0_25]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0_25]
        at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_25]
        at org.jboss.threads.JBossThread.run(JBossThread.java:122)
Caused by: java.lang.IllegalStateException: Cache container has been stopped and cannot be reused. Recreate the cache container.
        at org.infinispan.manager.DefaultCacheManager.assertIsNotTerminated(DefaultCacheManager.java:797)
        at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:389)
        at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:412)
        at org.jboss.as.clustering.infinispan.DefaultCacheContainer.getCache(DefaultCacheContainer.java:103)
        at org.jboss.as.clustering.infinispan.DefaultCacheContainer.getCache(DefaultCacheContainer.java:94)
        at org.jboss.as.clustering.infinispan.subsystem.CacheService.start(CacheService.java:78)
        at org.jboss.as.clustering.msc.AsynchronousService$1.run(AsynchronousService.java:86)
        ... 4 more

2015-05-27 18:20:54,710 ERROR [org.jboss.msc.service.fail] (MSC service thread 1-10) MSC000001: Failed to start service jboss.infinispan.web."my-application.war".config: org.jboss.msc.service.StartException in service jboss.infinispan.web."my-application.war".config: JBAS010291: Failed to add DIST_ASYNC my-application.war cache to non-clustered web cache container.
        at org.jboss.as.clustering.infinispan.subsystem.AbstractCacheConfigurationService.start(AbstractCacheConfigurationService.java:73)
        at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1948) [jboss-msc-1.2.2.Final.jar:1.2.2.Final]
        at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1881) [jboss-msc-1.2.2.Final.jar:1.2.2.Final]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0_25]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0_25]
        at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_25]

The way I initialize the cache is 

(def site-cache (agent
                  (immutant.caching/cache "site-cache" :ttl [ttl :seconds])
                  :error-mode :continue
                  :error-handler (fn [the-agent exception]
                                  (log/error "Exception in Site Agent " exception (string/join "\n" (.getStackTrace *e))))))

With using immutant 2.x my understanding is that the immutant.caching/cache should create the container if it doesn't exist. Is there a way for me to verify the cache container exists, and if not create it before I attempt to add a cache to it?

EDIT: This was being caused by an error with our deployment scripts. A hack had been done to deploy the same .war twice, which caused the application to not shut down properly causing the error.

jrahme
  • 253
  • 1
  • 12
  • I'm having difficulty duplicating your issue locally. Can you try it without an agent and see if the problem goes away? What version of WildFly are you using and is this in a cluster? – jcrossley3 May 27 '15 at 22:32
  • After further testing, I think I've ruled out the agent being a problem. Can you provide a small sample app that demonstrates the problem and I could build/deploy locally? Also, how are you [un]deploying the app: touching the war, using the jboss-cli, the web interface, or something else? – jcrossley3 May 28 '15 at 00:33
  • I am using Bamboo for my build system, and using the redeploy functionality of bamboo to redeploy it. Unfortunately my NDA stops me from being able to publish any part of my application. I can provide code snippets, but not any part of the application itself. – jrahme May 28 '15 at 12:42
  • I am using Wildfly 8.2.0 final, with JBoss 1.2.2 final and JBoss Modules 1.3.3 final – jrahme May 28 '15 at 13:04
  • I'm not familiar with Bamboo and I understand about the NDA. Would it be possible for you to try to replicate the issue using our feature-demo? I'm following these steps: https://github.com/immutant/feature-demo#in-wildfly. And when I touch that demo.war file to redeploy it, everything comes up fine. If you can figure out what to do differently to result in your issue, we can take it from there. Sorry for the trouble! – jcrossley3 May 28 '15 at 13:52

2 Answers2

2

It sounds like you may not be using the lein-immutant plugin to create your war file. The plugin adds some metadata to the war file to ensure the classpath is correct, among other things. It is definitely possible that a war file created by some other means, e.g. the lein-ring plugin, could deploy successfully the first time and leave the app server in an unpredictable state when undeployed.

jcrossley3
  • 11,576
  • 4
  • 31
  • 32
0

This was being caused by an error with our deployment scripts. A hack had been done to deploy the same .war twice for an application requirement. The hack stopped the application from shutting down cleanly when a redeployment happened, and was causing the issue with the cache.

jrahme
  • 253
  • 1
  • 12