1

My question might not have a clear answer so please let me know if what I am trying to do is unrealistic.

I have a Python script that runs several independent SQL statements. Due to timeout limits, waiting for the statements to finish execution is not an option. The statements are for maintenance and no output is expected. Is there a way to asynchronously trigger these?

For example, using psycopg2.cursor, I am expecting to do the following:

cursor.execute(sql_statement)
# Run next code block
Samarth
  • 119
  • 1
  • 1
  • 4
  • Run them in separate threads. – rd_nielsen Dec 08 '18 at 01:07
  • Possible duplicate of [How to use python to query database in parallel](https://stackoverflow.com/questions/37975904/how-to-use-python-to-query-database-in-parallel) – GMB Dec 08 '18 at 01:12
  • Run them in threads as rd_nielsen suggests, just be conscious of blocking on locks on tables / rows, which could cause your parallel implementation to single thread – seayak Dec 08 '18 at 01:47

1 Answers1

0

The recipe is threading. Use it in this way:

from threading import Thread
Thread(target=cursor.execute, args=(sql_statement,)).start()

It's worth noting that your program can not exit properly untill these threads are finished. If this behaviour is improper for you, you may pay attention to the subprocess module, which is capable of creating independently-running tasks.

reartnew
  • 247
  • 1
  • 9
  • Where is the `join` call on the `Thread`? You can't the way you have it chained off the return value - it's lost. – doug65536 Dec 02 '21 at 12:52
  • The question is about how to continue execution of a program waiting a query result in a handler - how I do understand non-blocking queries. – sergzach Jan 30 '22 at 19:20