I have a Python script that takes a list of names from the user, creates an SQL query string, and queries the database and puts the data into a dataframe using pandas.read_sql() method. However, I have noticed that if the list of people I want to query is very very long, my code never reaches the line coming after pandas.read_sql(), and for some reason, no exception is caught.
My query string looks like the following:
SELECT * FROM table1 WHERE
table1.name = 'Alice' OR
table1.name = 'Bob' OR
table1.name = 'Charlie' OR ...
where there could be hundreds of names included.
The Python code I am using looks like the following:
query_string = construct_string_above(names_list)
pymongoConnection = ...
try:
print("Trying")
df = pandas.read_sql(query_string, pymongoConnection)
except Exception as ex:
print(ex)
traceback.print_exc()
sys.exit(0)
print("Finishing")
My code always executes up until the try block. The program prints "Trying" but never prints "Finishing" nor does it return any errors. I checked, and the pymongoConnection object is correct. I also tried executing the query_string manually in MySQL and it works.
The issue must be at pandas.read_sql() and even here, the issue only arises when the size of the query string exceeds a certain threshold, otherwise the code finishes correctly. Are there any limitations to the use of pandas.read_sql(), such as maximum size of input string or maximum WHERE clauses or maximum amount of data returned to the dataframe? Because I cannot think of anything else that might cause the problem.