I have the problem that my django application accumulates postgres connections over time. It seems that about every 30min a new connection is established and the old connections don't close (see screen). As max connections is set to 100 after some time all connections are blocked.
Does anyone know what is causing this problem?
I discovered this after I integrated some celery tasks. So I am quite sure that it is related to celery.
So I tried to close the connection manually after every Task using after_return method:
from django.db import connection
class DBTask(Task):
abstract = True
def after_return(self, *args, **kwargs):
connection.close()
@task(name='example', base=DBTask)
def example_task(value):
# do some stuff
But this also doesn't help.
Maybe I am totally wrong and it isn't related to celery at all.
My database configuration:
DATABASES = {
'default': {
'ENGINE': 'django.contrib.gis.db.backends.postgis',
'NAME': 'production',
'USER': 'production',
'HOST': 'some.host',
'CONN_MAX_AGE': 0,
},
}
Installed packages:
- django 1.8.9
- pyscopg2 2.6.1
- celery 3.1.20
- django-celery 3.1.17
The app is deployed at webfaction (maybe this helps)
I have also seen this question, but setting CONN_MAX_AGE: 0 didn't help.
Update:
Tried adding connection.close() at the end of each celery task, but the number of connection is still increasing.
Update 2:
Tried adding connection.close() at the top of the celery file, but this didn't help either.
Update 3:
Here is the code I am actually using in the celery tasks:
celery_tasks.py
@task(name='push_notifications', base=DBTask)
def push_notifications_task(user_id):
user = CustomUser.objects.get(id=user_id)
PusherAPI().push_notifications(user)
connection.close()
models.py
class PusherAPI(object):
def push_notifications(self, user):
from .serializers import NotificationSerializer
self.pusher.trigger(
'user_%s' % user.slug,
'notifications',
NotificationSerializer(user).data
)
serializers.py
class NotificationSerializer(object):
def __init__(self, user=None):
if user is None:
self.user = get_current_user()
else:
self.user = user
@property
def data(self):
# get notifications from db
notifications = self.user.notifications.unread()
# create the notification dict
...
return note_dict
The only db-queries are in CustomUser.objects.get(id=user_id) and notifications = self.user.notifications.unread()
