1

I'm running a celery(4.1.0) task with django(1.11.6) using psycopg2(2.7.3), getting these errors after 5-6 hour of deploying the app on production. One of the related issue on Django issue tickets. Tried various solution of reconnecting the connection using try and except, handling InterfaceError. Any help will be appreciated. Checked and Applied some of the solutions, but nothing seems to be working, Celery Worker Database Connection Pooling

Traceback

InterfaceError: connection already closed
  File "billiard/pool.py", line 358, in workloop
    result = (True, prepare_result(fun(*args, **kwargs)))
  File "celery/app/trace.py", line 537, in _fast_trace_task
    uuid, args, kwargs, request,
  File "celery/app/trace.py", line 482, in trace_task
    I, _, _, _ = on_error(task_request, exc, uuid)
  File "celery/app/trace.py", line 330, in on_error
    task, request, eager=eager, call_errbacks=call_errbacks,
  File "celery/app/trace.py", line 164, in handle_error_state
    call_errbacks=call_errbacks)
  File "celery/app/trace.py", line 212, in handle_failure
    task.on_failure(exc, req.id, req.args, req.kwargs, einfo)
  File "telemetry/tasks.py", line 29, in on_failure
    save_failed_task(self, exc, task_id, args, kwargs, einfo)
  File "telemetry/celery_failure.py", line 46, in save_failed_task
    existing_task_first = existing_task.first()
  File "django/db/models/query.py", line 564, in first
    objects = list((self if self.ordered else self.order_by('pk'))[:1])
  File "django/db/models/query.py", line 250, in __iter__
    self._fetch_all()
  File "django/db/models/query.py", line 1118, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File "django/db/models/query.py", line 53, in __iter__
    results = compiler.execute_sql(chunked_fetch=self.chunked_fetch)
  File "django/db/models/sql/compiler.py", line 882, in execute_sql
    cursor = self.connection.cursor()
  File "django/db/backends/base/base.py", line 254, in cursor
    return self._cursor()
  File "django/db/backends/base/base.py", line 231, in _cursor
    return self._prepare_cursor(self.create_cursor(name))
  File "django/db/utils.py", line 94, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "django/utils/six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "django/db/backends/base/base.py", line 231, in _cursor
    return self._prepare_cursor(self.create_cursor(name))
  File "django/db/backends/postgresql/base.py", line 220, in create_cursor
    cursor = self.connection.cursor()
nik_m
  • 11,825
  • 4
  • 43
  • 57
untwal
  • 1,017
  • 1
  • 9
  • 17
  • 2
    Not really but the main cause of the error was longer running transactions, Mostly blocking ones. – untwal Mar 05 '18 at 11:52

0 Answers0