0

Background: I'm converting python script which takes values from django-db to be multi-threaded using Process. I might missed something (quite new in django) but got "DatabaseError: SSL error: decryption failed or bad record mac" So I've closed the connection (according to this)... but than got "DatabaseError: SSL connection has been closed unexpectedly".

I decided to try psycopg2 connected & closed the connections in the locations of the django version. All worked well.

Issue: when I've tested the solution I saw that the same query return different values!

My query is very simple: "SELECT DISTINCT "someId" FROM aTable WHERE "date" < 'date_in_the_past';"

Please advice..

UPDATE: I can't show the complete code.. I've changed names etc' but flow is 100% the same. The lanchAll method launches process for every t_id, when I search with django.db connection I get 3 distinct values, when make it with psycopg2 I get 1 value. In runtime this is called with in_sim which is a date in the past. In lanchAll the django.db connection is marked as comment right next to the psycopg2...

from multiprocessing import Process
import sys
from numpy import *
from itertools import izip_longest
from datetime import * 
import psycopg2
from appName import settings
from django.core.management import setup_environ
from django.core.exceptions import ObjectDoesNotExist

import argparse
import pdb
setup_environ(settings)

from appName.models import *
from appName.aClass import *
from django.db import connection

def lanchAll(in_sim):
    p = 0
    try:
        con = psycopg2.connect(database=DB_NAME, user=DB_USER, password=DB_PASS,
                        host=DB_IP)
        cursor = con.cursor()
        # cursor = connection.cursor()
        if in_simolationDate is None:
            query = 'SELECT DISTINCT "t_id" FROM appName_tableNAme '\
                'WHERE "date" > now() - interval \'%s seconds\';'        
            cursor.execute(query, [TIME_WINDOW])    
        else:
            query = 'SELECT DISTINCT "t_id" FROM appName_tableNAme '\
                'WHERE "date" < %s ;'        
            cursor.execute(query, [in_sim])    

    except ObjectDoesNotExist as e:
        con.close()
        # connection.close()
        print((str('DB Error: %s\n' % e)))
        return -3
    if cursor.rowcount < 1:
        con.close()
        # connection.close()
        print ("offline!")
        return -4

    for row in onlineTagsCursor:
        t_id = row[0]   
        print "currently lunching tagId:" + str(t_id)
        processInstance = Process(target=launchThread, args=(t_id, p,in_sim))
        processInstance.start()

    con.close()
    # connection.close()


def launcher(in_id, in_p,in_sim):
    #I'll add it if you fell that it is needed.. 


#the model for the table     
class tableNAme(models.Model):
    m_id    = models.IntegerField()
    sc_id   = models.IntegerField()
    t_id    = models.IntegerField()
    r       = models.IntegerField()
    l       = models.IntegerField()
    g       = models.IntegerField(0)
    p       = models.IntegerField(0)
    date    = models.DateTimeField(auto_now=True)
Community
  • 1
  • 1
user2495766
  • 123
  • 3
  • 15

1 Answers1

0

I suspect you are forking after the connection has been created. Libpq connections (the ones used by psycopg) don't support fork.

Make sure the new processes create a connection of their own, they cannot use one created by the parent process.

piro
  • 13,378
  • 5
  • 34
  • 38