RabbitMQ keeps connections running for a long time

Question

I am using a hosted RabbitMQ provider i.e CloudAMQP and running a django app server on heroku. In local environment everything seems to work fine but on Heroku where CloudAMQP has a limit on the number of simultaneous connections to the broker, the connections seem to persist forever eventually resulting in the app running out of available open connections. This happens even when the tasks associated with earlier connections have executed by the subscriber (i.e a heroku worker).

If you have been able to successfully use CloudAMQP with Heroku and django celery, can you post the things that worked for you?

Answer 1

Was able to fix this by adding BROKER_POOL_LIMIT=0 to my production settings.

Answer 2

BROKER_POOL_LIMIT=0 means any part of your setup that needs a connection will establish it, do its thing and disconnect

while this helps with idle setups, you will find yourself sporadically hitting that 6 connection limit during periods of higher traffic, because you simply need more than 6 connections for all your stuff to talk to cloudamqp

check out heroku celery exceeding connection limit for an explanation of why and how to fix it