Singleton with cluster

Question

I have a such class:

@Singleton
public class RecoveryWorker {
    @Schedule(minute = "*/1", hour = "*", persistent = false)
    public void run() {
        // heavy db queries ~ 2sec
        ...
    }
    ...
}

There are 2 wildfly instances which has this singleton and works with the same oracle DB

Sometimes I had error:

RROR [org.jboss.as.ejb3] (EJB default - 10) JBAS014120: Error invoking timeout for timer: [id=66ed65c4-3a2d-4343-870d-5f7a46a7742c timedObjectId=com.package.Worker auto-timer?:true persistent?:false timerService=org.jboss.as.ejb3.timerservice.TimerServiceImpl@222950e6 initialExpiration=null intervalDuration(in milli sec)=0 nextExpiration=Wed Jun 15 00:01:00 CEST 2016 timerState=IN_TIMEOUT info=null: javax.ejb.ConcurrentAccessTimeoutException: JBAS014373: EJB 3.1 PFD2 4.8.5.5.1 concurrent access timeout on org.jboss.invocation.InterceptorContext$Invocation@ead4bd3 - could not obtain lock within 5000MILLISECONDS

I want to know how this block is happened (Oracle has a mechanize to block table or something else?) What is the best practice to run one singleton among a few instances in one time?

Take a look in http://stackoverflow.com/questions/27952769/clustered-singleton-using-wildfly and https://docs.jboss.org/author/display/WFLY10/HA+Singleton+Features if you are using Wildfly 10 — Federico Sierra, Jul 08 '16 at 13:36

score 0 · Answer 1 · answered Jul 11 '16 at 07:13

The @Singleton is only singleton for the JVM, there is no guarantee that it run only on one instance at a time.

The JBAS014120 is because of the @Singleton contract, your timer is every minute. In your case I suppose that sometimes the "heavy db query" can run longer and another singleton will be started, as you don't have lock permission set to READ every invocation to the singleton will be blocked by contract.

If you need a Cluster wide singleton you should check the quickstarts. I've written the cluster-ha-singleton which is exact the example for your use-case. https://github.com/wildfly/quickstart It will elect one node and start the schedule only on one of the cluster nodes, in case of the node crashed it will failover.

Singleton with cluster

1 Answers1