I have typical client-server system with ActiveMQ in the middle. Client puts request into queue and waiting for response in temporary queue which is created for each request. ActiveMQ and my server components run on the same workstation. This system worked fine for 21 days but then all request started to fail permanently (not a single request was processed successfully). I see the following in server logs:
2015-08-12 11:35:46,242 WARN Execution of JMS message listener failed, and no ErrorHandler has been set. [DefaultMessageListenerContainer][org.springframework.jms.listener.DefaultMessageListenerContainer#1-23]
javax.jms.InvalidDestinationException: Cannot publish to a deleted Destination: temp-queue://ID:<QUEUE_ID>
at org.apache.activemq.ActiveMQSession.send(ActiveMQSession.java:1841) ~[activemq-all-5.11.1.jar:5.11.1]
at org.apache.activemq.ActiveMQMessageProducer.send(ActiveMQMessageProducer.java:288) ~[activemq-all-5.11.1.jar:5.11.1]
at org.apache.activemq.ActiveMQMessageProducer.send(ActiveMQMessageProducer.java:223) ~[activemq-all-5.11.1.jar:5.11.1]
at org.apache.activemq.ActiveMQMessageProducerSupport.send(ActiveMQMessageProducerSupport.java:241) ~[activemq-all-5.11.1.jar:5.11.1]
Request is usually performed pretty fast (~1 ms) so it hardly possible that timeout occured on client side. Same time I see in activemq logs several errors:
//It is my first server application
2015-08-12 11:35:46,242 | WARN | Transport Connection to: tcp://127.0.0.1:64322 failed: java.net.SocketException: Software caused connection abort: recv failed | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ Transport: tcp:///127.0.0.1:64322@61616
//It is my second server application
2015-08-12 11:35:46,242 | WARN | Transport Connection to: tcp://127.0.0.1:64323 failed: java.io.EOFException | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ Transport: tcp:///127.0.0.1:64323@61616
//Client (maybe he just closed application)
2015-08-12 11:36:20,311 | WARN | Transport Connection to: tcp://<client_ip>:60550 failed: java.net.SocketException: Connection reset | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ Transport: tcp:///<client_ip>:60550@61616
2015-08-12 11:36:20,311 | INFO | temp-queue://ID:<QUEUE_ID> purged of 0 messages | org.apache.activemq.broker.region.Queue | ActiveMQ BrokerService[localhost] Task-141551
All applications was stopped and started again only after 2 hours. Since that no more errors by now. I am trying to reproduce this but with no success. Can anyone give me a clue what could be the cause of such behaviour?
My environment:
Windows Server 2008 (Windows Firewall disabled)
Java 1.8.0_20
ActiveMQ 5.11.1