1

I'm using Apache Camel, SpringBoot and WMQ in my application to connect to an external service. When the connection to this service fails e.g. when the service is down, I'm rolling back the message to the input queue using JMS transaction. However, the messages are re-consumed instantly after redelivery, which is not what I want. I'd like the messages to be consumed again but after say every 30 minutes giving the external service enough time to come back up.

Is there a way to configure this with JMS transactions and adding Hystrix CircuitBreaker (or any other library) in Apache Camel?

Thanks

JMS Configuration:

@Bean
public UserCredentialsConnectionFactoryAdapter connectionFactoryAdapter(final MQQueueConnectionFactory connectionFactory) {
    UserCredentialsConnectionFactoryAdapter connectionFactoryAdapter = new UserCredentialsConnectionFactoryAdapter();
    connectionFactoryAdapter.setTargetConnectionFactory(connectionFactory);
    return connectionFactoryAdapter;

}

@Bean
public CachingConnectionFactory cachingConnectionFactory(
    final UserCredentialsConnectionFactoryAdapter connectionFactoryAdapter) {
    CachingConnectionFactory cachingConnectionFactory = new CachingConnectionFactory();
    cachingConnectionFactory.setCacheConsumers(true);
    cachingConnectionFactory.setCacheProducers(true);
    cachingConnectionFactory.setReconnectOnException(true);
    cachingConnectionFactory.setSessionCacheSize(1);
    cachingConnectionFactory.setTargetConnectionFactory(connectionFactoryAdapter);
    return cachingConnectionFactory;
}

@Bean
public JmsTransactionManager jmsTransactionManager(final CachingConnectionFactory cachingConnectionFactory) {
    JmsTransactionManager jmsTransactionManager = new JmsTransactionManager();
    jmsTransactionManager.setConnectionFactory(cachingConnectionFactory);
    return jmsTransactionManager;
}

@Bean
public SpringTransactionPolicy springTransactionPolicy(final PlatformTransactionManager jmsTransactionManager) {
    SpringTransactionPolicy policy = new SpringTransactionPolicy();
    policy.setTransactionManager(jmsTransactionManager);
    policy.setPropagationBehaviorName("PROPAGATION_REQUIRED");
    return policy;
}

@Bean
public JmsConfiguration jmsConfiguration(final CachingConnectionFactory cachingConnectionFactory,
    final JmsTransactionManager jmsTransactionManager) {
    JmsConfiguration config = new JmsConfiguration();
    config.setConnectionFactory(cachingConnectionFactory);
    config.setTransactionManager(jmsTransactionManager);
    config.setTransacted(true);
    config.setConcurrentConsumers(4);
    config.setMaxConcurrentConsumers(8);
    return config;
}

@Bean
public JmsComponent jmsComponent(final JmsConfiguration jmsConfiguration) {
    JmsComponent component = new JmsComponent();
    component.setConfiguration(jmsConfiguration);
    component.setPreserveMessageQos(true);
    component.setDeliveryPersistent(false);
    return component;
}

@Bean
public RedeliveryPolicy redeliveryPolicy() {
    RedeliveryPolicy redeliveryPolicy = new RedeliveryPolicy();
    redeliveryPolicy.setMaximumRedeliveries(2);
    redeliveryPolicy.setMaximumRedeliveryDelay(10000);
    redeliveryPolicy.setRedeliveryDelay(10000);
    return redeliveryPolicy;
}

Camel Routes (using SpringRouteBuilder):

errorHandler(
        transactionErrorHandler()
            .maximumRedeliveries(redeliveryPolicy.getMaximumRedeliveries())
            .redeliveryDelay(redeliveryPolicy.getRedeliveryDelay())
            .maximumRedeliveryDelay(redeliveryPolicy.getMaximumRedeliveryDelay()));

onException(MyException.class)
        .markRollbackOnly()
        .redeliveryPolicy(redeliveryPolicy)
        .useExponentialBackOff()
        .handled(true)

from("jms:queue:myQueue")
        .id("myRoute")
        .autoStartup(true)
        .tracing()
        .transacted()
        .process(bean)
        .stop();
RGB314
  • 177
  • 1
  • 4
  • 15
  • can't you control that with redeliveryDelay, by setting it to 30 minutes – pvpkiran Nov 15 '17 at 11:56
  • 2
    Look at what your JMS message broker supports in terms of transactional redelivery. They should have options where you can customize the delay and number of attempts etc. In other words - this is outside the control/domain of Apache Camel but is handled by the JMS message broker. This is NOT the same as Camel error handling with redelivery. They are not the same. – Claus Ibsen Nov 15 '17 at 14:33
  • @ClausIbsen The version of the message broker being used is an old one, which does not support delay for redelivery. Is there a workaround like setting a custom header and checking the last processed timestamp for that message or exchange ID? Or as suggested above, make the redeliveryDelay large enough, maxRedelivery as -1 and use exponential backoff with a multiplier? – RGB314 Nov 15 '17 at 14:59
  • You don't mention what version of IBM MQ client jar files you are using or what version the MQ server is at (you do say it is old), if it is v8.0 or higher, IBM MQ implements JMS 2.0 which has a Delayed Delivery feature. One suggestion is if your external service is down you put the message back to your queue using the Delayed Deliver feature, MQ would then old it in a "delayed delivery queue" and send it on to your normal input queue at the appropriate time. – JoshMc Nov 16 '17 at 07:24
  • 1
    These three questions and answer may provide some help: "[How do I limit the amount of times a JMS DefaultMessageListenerContainer will retry a message?](https://stackoverflow.com/questions/10552514)", "[Implementing retry logic in Ibm Websphere MQ with spring](https://stackoverflow.com/questions/34342579)", and "[Configuring a 'retry delay' in MQ Series](https://stackoverflow.com/questions/1322878)" – JoshMc Nov 16 '17 at 07:26
  • Another option is to configure a back out threshold and back out queue on the queue manager, if you configured the threshold to 1 your JMS app would move the message to the back out queue when it gets the message a second time. You could then have another consumer that you schedule to run every 30 minutes that would move messages from the back out queue back to the input queue. This feature has been around in MQ for many versions (at least back to 5.3). – JoshMc Nov 16 '17 at 07:30
  • @JoshMc To be honest, I'm not sure about the version being used, it is either 6 or 7 but I'm sure that it is not using JMS 2.0. Unfortunately, configuring at the broker level is not an option at the moment. Presently, I'm trying to use the redelivery policy to increase the time gap between retries, seems to work for now. Would this approach have any drawbacks in your opinion? I'll also have a look at the links you mentioned above – RGB314 Nov 16 '17 at 10:47
  • You can look at any of your com.ibm.mq.*jar files to see what version they are from on the client side. On linux you can do the following `unzip -p com.ibm.mq.jar META-INF/MANIFEST.MF|grep Implementation-Version`, The output will be: `Implementation-Version: x.x.x.x - pxxx-xxx-YYMMDD` where the first x.x.x.x is the version number. On window you could do something similar with zip or even just renaming one of the jar files as .zip and opening it as a compressed folder. For the server you would likely need to ask the MQ administrator. – JoshMc Nov 16 '17 at 17:04
  • I do not know enough about Camel or Spring to give you an opinion on your approach. How does does this function at the messaging layer, does it just delay the time before the app does a back out of the message to the original queue? – JoshMc Nov 16 '17 at 17:09
  • The version of MQ is 7.5.0.5 which does not have JMS 2.0 implementation. Regarding the redelivery policy, my approach just adds a 15 minute interval between retries. Ideally if the broker supported, the message would be instantly rolled-back to the queue and consumed after 15 minutes. In this case, number of retries could have been 0 as well but presently it's -1 i.e. it will be retired every 15 minutes until it succeeds. – RGB314 Nov 17 '17 at 11:05
  • 1
    Per the "Camel in Action" book, pg 486, the RetryPolicy that you use in your code should NOT be used with transactions. What I understand is that those retries simply repeat the same failing step, repeat at the point of error. They do not roll back the transaction (so transaction is open during the entire retry period), and the message is also not put into its original state and reprocessed from the beginning. – Christoph Oct 17 '20 at 12:35

0 Answers0