8

I have a problem by using Apache Camel in combination with Oracle Advanced Queues and JMS.

It's about an application to distribute messages. Messages are received and enqueued in Oracle Advanced Queues with the help of Camel. Then they are consumed with Camel and forwarded to the target system. For the case that the message delivery fails, there is an retry count defined in the Advanced Queue, so that the message delivery is repeated.

If Camel now dequeue a message and sends it to a target system that is not available, either a HttpOperationFailedException or NoSuchEndpointException is thrown. These are caught and there is a rollback performed.

At this point, the expectation is that the message delivery will be retried as often as defined in the Retry Count and then moved to an exception queue. However, what is happening is that the next message in the queue is send.

Because the contents of the messages are partially dependent on each other, they must be processed sequentially.

I think that there is an misconfiguration in the usage of the JMS Library, but I'm not sure and have found nothing I could influence this behavior.

The used JMS library is Oracle AqApi v 11.2.0.3.

Here is the code for the Camel route:

from("jms-camel-component:myComponent.AQ?jmsMessageType=Text").routeId("deliveryToTarget")
        .transacted()                
        .setExchangePattern(ExchangePattern.InOut)                        
        .setHeader(Exchange.HTTP_QUERY, constant("throwExceptionOnFailure=false"))
        .setHeader(Exchange.CONTENT_TYPE, constant("application/xml; charset=UTF-8"))
        .doTry()
            .recipientList(header("endpointTarget"))
            .endDoTry()
            .process(ResponseProzessor.getInstance())
            .log("Message was delivererd.")
        .doCatch(HttpOperationFailedException.class, NoSuchEndpointException.class)    
            .process(ResponseProzessor.getInstance())
            .log("Error occured.")
            .rollback()
        .end();

Here is the JmsComponent configuration:

JmsComponent jmsComponent = new JmsComponent(scc);
jmsComponent.setConnectionFactory(connectionFactory);
jmsComponent.setTransactionManager(tm);
jmsComponent.setMaxConcurrentConsumers(1);            
jmsComponent.setMaxMessagesPerTask(1);                  
jmsComponent.setIncludeSentJMSMessageID(true);

Thank you in advance for your help!

UPDATE

I think, I've found the reason for the described behavior. There is a delay configured on the Advanced Queue. As long as the delay lasts, the next message from the queue is dequeued. The messages are not dequeued randomly, they are dequeued according to the priorities.

I really think this is something that has to be configured on the consumer. Is there any trick to configure the camel-jms-component to consume the first message from queue as long as it's not commited or moved to the exception queue? I didn't find an option to configure it directly on camel...

The Dodo
  • 147
  • 8
  • 1
    In JMS in general, message order is not guaranteed. In ActiveMQ, if you have only a single consumer, and client-side redelivery, you're pretty safe, but I don't know about Oracle AQ... – GeertPt Sep 16 '15 at 14:48
  • I would be interested in a sollution for Weblogic Application Servers if there is no universal sollution for this problem. – elfwyn Sep 23 '15 at 11:44
  • The (clean) solution to this is always server side as the client (should/) doesn't know the messages in the queue. Depending on your QoS you'd need to read all the messages in the queue to figure out which message you want first - which, depending on your options (configuration wise) could still be an option. I'd check my answer below / the configuration on the server side before you do implement this on the client side. HTH – PhilW Sep 24 '15 at 08:32
  • In hinsight the problem seems to have been a delay set on the Advanced Queue DB-side. This caused the next message to be consumed by the Camel Framework before the previous transaction was rolled back cleanly in case of an error. It would be nice though to know if that is the right thing to happen under these circumstances or an oversight by the Oracle AQ Implementation. – elfwyn Sep 28 '15 at 07:05
  • @PhilW: Does that mean that I have to put a message broker like Apache ActiveMQ between Camel and the database? As far as I understood, those message broker serve the same purpose as the Oracle Advanced Queues. It's not more than another queue. Is that correct? So isn't there one obsolete? – The Dodo Sep 29 '15 at 06:13

1 Answers1

3

I'm not a Oracle AQ expert, but as far as I can see this is a setting on the queue, not on the client side.

The sort_list parameter determines the order in which messages are dequeued. You cannot change the message sort order after you have created the queue table

from: http://docs.oracle.com/cd/B19306_01/server.102/b14257/aq_admin.htm

You most likely have ENQ_TIME or COMMIT_TIME set - which might already satisfy your needs.

And, of course, your consumer must be the only one.

PhilW
  • 741
  • 6
  • 23
  • In my experience with AQ, the Java is usually simple boiler plate and real configuration happens to queue in the DB. Might also want to look into message grouping for dependent messages http://docs.oracle.com/cd/B10501_01/appdev.920/a96587/qsample.htm#81528 – CPrescott Sep 23 '15 at 14:41
  • You're right. The real configuration takes place in the database. It's possible to configure a delay for the queue but it seems that then the message will only be blocked for the time of the delay, which means that jms starts to consume the next message. Thanks for the link but message grouping seems not to be an solution because the messages are not send in the same transaction. – The Dodo Sep 29 '15 at 06:22
  • Thank you for helping us with our problem. Our initial problem was a timeout set in the AQ which lead to out-of-order-processing of queue messages. Although your tip wasn't the essential lead to our sollution I find it prudent to award you the bounty. Thank you! – elfwyn Sep 30 '15 at 12:28