Python: decryption failed or bad record mac when calling from Thread

Question

I'm getting a "decryption failed or bad record mac" error in this code-fragment:

conn = psycopg2.connect(...)
cursor = conn.cursor()
cursor.execute("SELECT id, ip FROM schema.table;")
rows = cursor.fetchall()
cursor.close()
conn.commit()
conn.close()

This is called in the run() method of a Thread, several times in a while(True) loop. I'm just opening a connection to my PostgreSQL database using the psycopg2 driver.

Any idea of how safe is opening db connections into Threads in Python? I don't know what is raising this error.

Is the exception raised on every iteration of the loop? Which line is the one that actually raises it? — Wieland, Dec 18 '13 at 15:27
Hello, it's raised at cursor.execute("SELECT id, ip FROM schema.table;"). It's not being thrown in every single iteration, it does few iterations right, then raises the error once and stop the execution. — Iván, Dec 18 '13 at 15:35
Ok, looks like I've fixed the problem. I was creating too many connections and seems I was running out of memory or something. conn = None after conn.close() does the trick. Thanks for your response. — Iván, Dec 18 '13 at 16:00
You can post your comment as an answer and even accept it after a while. In that case people can see from the question that the issue has been resolved. You will get points if people click the +1 on question or answer (like this). — Maarten Bodewes, Dec 18 '13 at 17:03
I was facing a similar problem while trying to run Multiprocessing and Postgresql in Python. I noticed that I was using the same connection in all spawned process and hence it was failing. I passed engine to different processes and then created new connections in their respective processes. It worked! — p699, Dec 01 '18 at 17:16

score 6 · Accepted Answer · answered Dec 19 '13 at 09:27

6

Ok, looks like I've fixed the problem. I was creating too many connections and seems I was running out of memory or something. I gathered all the queries and do cursor.execute(...) once with a huge query, instead performing hundreds of small queries/connections.

conn = psycopg2.connect(...)
cursor = conn.cursor()
cursor.execute("SELECT id, ip FROM schema.table;")
rows = cursor.fetchall()
cursor.close()
conn.commit()
conn.close()
conn = None

answered Dec 19 '13 at 09:27

Iván

552
8
16

5

Just a side note: A `with conn as cursor:` would incorporate the corsor creation, the `cursor.close()` and the `conn.commit()` into one. – glglgl Dec 19 '13 at 10:00

score 0 · Answer 2 · answered Jul 08 '18 at 08:07

The cause of this issue could be, there were too many processes(multi) were trying to access PostGres, it was not able to handle that. I was using Django & PostGres in BeanStalk.

Adding 'OPTIONS': {'sslmode': 'disable'} in the database config helped.

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql_psycopg2',
        'NAME': ...
        'USER': ....
        'PASSWORD': ...
        'HOST': ....
        'PORT': '5432',
        'OPTIONS': {
           'sslmode': 'disable',
        }
    }
}

Python: decryption failed or bad record mac when calling from Thread

2 Answers2

Linked