2

I am new to python-threading programming. I want to execute my Database Queries in parallel. Assuming two separate queries, how to run these in parallel to query same database, and also wait for both results to return before continuing the execution of the rest of the code?

from threading import Thread, Lock

class DatabaseWorker(Thread):

    def __init__(self, connection, query, paramDictFilter, key, query_mode,  lstResultData):

        Thread.__init__(self)

        self.connection = connection
        self.query = query
        self.paramDictFilter = paramDictFilter
        self.key = key
        self.query_mode = query_mode
        self.lstResultData = lstResultData
        self.lock = Lock()

    def run(self):

        resultData = dict()
        try:
        with self.lock:
            with self.connection.cursor('dict') as cur:
                cur.execute(self.query, self.paramDictFilter)

                if self.query_mode == 0:
                    resultData[self.key] = cur.fetchone()

                else:
                    resultData[self.key] = cur.fetchall()

                cur.close()

        except Exception as e:
            print(e)

        self.lstResultData.append(resultData)

Outside Class:

connV = Database.connect_vertica_dbs()

qryDefaultSection = """ """

qryBusinessAddress = """ """

workerBusinessAddress = DatabaseWorker(connV, qryBusinessAddress, paramDictFilter, "businessAddr", 1, lstResultData)

workerDefaultSection = DatabaseWorker(connV, qryDefaultSection, paramDictFilter, "defaultHistory", 1, lstResultData)

workerBusinessAddress.start()
workerDefaultSection.start()

delay = 1

while len(lstResultData) < 2:
     time.sleep(delay)

workerBusinessAddress.join()
workerDefaultSection.join()


connV.close()

This run only for the first time and after that it goes to infinity loop and have different attribute errors.

SuperShoot
  • 9,880
  • 2
  • 38
  • 55
Shahbaz Shabbir
  • 49
  • 1
  • 1
  • 4
  • Can you reformat the code please? Please see my answer here and let me know if it is helpful: https://stackoverflow.com/questions/54927462/how-to-parallelize-complicated-for-loops/54927584#54927584 – Scott Skiles Apr 10 '19 at 14:35

0 Answers0