0

I'm having some trouble converting some working code to a function. The first code sample is running without issue, but the function below just hangs. A little debugging using print showed that the function moves all the way through the cursor to the final record, appending it to the list, but the the program hangs and won't exit.

Cursor is part of the cx_Oracle module. The intent is to query an oracle db, then create a list. I have tested the original code on several queries and had no problems (max return is about 15000 rows). At this point I can make the code work using the original format, but I'd like to know what I might be doing wrong in the function.

Working code:

cursor = db.cursor()
cursor.execute(mysqlexp)
for row in cursor:
    myList.append(row)
cursor.close()

Function (not working):

def sqlToList(listname, sqlexp):
    cursor = db.cursor()
    cursor.execute(sqlexp)
    for row in cursor:
        listname.append(row)
        #print statement here indicates that final record appends
        #but then the program stops responding
    #print statement here never appears (indicating for loop hasn't exited?)
    cursor.close()
sqlToList(myList, mysqlexp)
martineau
  • 119,623
  • 25
  • 170
  • 301
skjold
  • 1
  • 1
  • Side-note: A loop that does nothing but repeatedly `append` can be removed in favor of `listname.extend(cursor)`. Of course, if the `cursor`'s iterator is blocking instead of raising `StopIteration` when it finishes, it would have the same problem as the loop. I don't know of any obvious reason why that would happen at all though, let alone differ between in and out of functions. – ShadowRanger Aug 12 '16 at 15:30
  • Question: Are you flushing `sys.stdout` after printing, or printing to `sys.stderr` to ensure you see all `print` output? Odds are `stdout` line buffering should work, but if you're running the script in a wrapper or piping the output through anything, it might switch to block buffering and your `print` debugging would give you a false impression of where the problem occurs. I could potentially see a bug related to the `cursor`s `__del__` that you don't see in the unwrapped case because the `cursor` is not cleaned until program exit there, while the function would clean up on stack unwind. – ShadowRanger Aug 12 '16 at 15:32
  • Just for the debug sake, can you get cursor.rowcount and break the for loop forcefuly once your reach the rowcount? Just to check if the script is responding – be_good_do_good Aug 12 '16 at 15:41
  • Also cannot think of anything that would cause the same query to block when used in a function. The only thing that comes to mind is the query is just taking so long to run. Are the two queries identical and produce the same results? BTW - for debugging can use the built in debugger with `python -m pdb `, then add break point at the line you want to stop at with `break `. `continue` will run program to break point and you can then inspect all variables, step through rest of code etc. Debuggers are awesome :) – danny Aug 12 '16 at 15:42
  • @ShadowRanger, I'm admittedly not super fluent in python so some of what you have suggested will take me some time to process and test. – skjold Aug 12 '16 at 16:01
  • @skjold: Basically, I'm saying that, to confirm the point of failure, make sure to `import sys` at the top of your file, and follow each of your debugging `print`s with a line that consists solely of `sys.stdout.flush()`. If you're on modern Python 3, you can simplify further, avoiding the import and call and just passing the argument `flush=True` to the `print` function. – ShadowRanger Aug 12 '16 at 16:08
  • @danny, thanks for the suggestions. I have a few queries set up to run with a user input variable for a unique identifier. I've run it on several id's without issue (outside the function) and it only takes a few seconds for the entire script to run, including a few other primitive functions (create csv, etc) – skjold Aug 12 '16 at 16:09
  • @be_good_do_good , it appears that cursor.rowcount only returns the number of rows that have been read from the cursor so far. Immediately after cursor.execute() the rowcount is 0. After the for loop ends it will contain the max row, so I don't think it will work as an iterator. – skjold Aug 12 '16 at 16:54
  • @ShadowRanger , thanks for the simplification. I tried sys.stdout.flush() and received the same results. It stops after the last record – skjold Aug 12 '16 at 17:03
  • For debugging purposes, you could fetch all the rows using "cursor.fetchall()", then loop through the ordinary list to check for side-effects in your actual code. – swstephe Aug 13 '16 at 15:12

0 Answers0