Questions tagged [read-sql]
45 questions
7
votes
4 answers
`pd.read_sql(sql, engine)` raises NotImplementedError: This method is not implemented for SQLAlchemy 2.0
I tried to create a pandas DataFrame directly from my sqlserver database using an sqlalchemy engine:
engine = create_engine(URL_string, echo=False, future=True)
query_string = "..."
dt = pd.read_sql(query_string, engine)
But this raises this…
Hajar Razip
- 449
- 5
- 11
3
votes
1 answer
How can I use pandas.read_sql on an async connection?
I am trying to do the asynchron equivalent of
engine = create_engine('sqlite:///./test.db')
stmt = session.query(MyTable)
data = pd.read_sql(stmt, engine)
but it fails with the error AttributeError: 'AsyncConnection' object has no attribute…
Bjoern
- 171
- 1
- 11
3
votes
1 answer
Pandas' `read_sql` creates integer columns when reading from an Oracle table which has number columns with decimal points
I have a Oracle table with columns of type VARCHAR2 (i.e. string) and of type NUMBER (i.e. a numeric value with a fractional part). And the numeric columns contain indeed values with decimal points, not integer values.
However when I read this…
halloleo
- 9,216
- 13
- 64
- 122
2
votes
1 answer
Read SQL from AWS Athena with Polars
I want to read from AWS Athena with polars. Is this possible? Before I used pandas:
import pandas as pd
pd.read_sql(SQL_STATMENT, conn)
I found this User Guide: https://pola-rs.github.io/polars-book/user-guide/howcani/io/read_db.html
where Athena…
Florian
- 21
- 1
2
votes
1 answer
Is there a work around for Pandas 2.0 deprecation of pyodbc for obscure databases?
According to Gord Thompson here: Getting a warning when using a pyodbc Connection object with pandas
pyodbc is not deprecated and I agree, for databases with a SQLAlchemy dialect. But I am using an obsure database - Providex on which Sage 100 ERP is…
dain
- 63
- 5
2
votes
1 answer
Python Snowflake - Large Dataset Extraction - Stop Iteration / Hanging Error
When I extract the small dataset from Snowflake, it works with no issues. But when I try to extract the large dataset from Snowflake using Python SNOW connector, it throws an Operational Error.
Any help is appreciated.
Error:
OperationalError:…
Bala
- 31
- 3
1
vote
1 answer
I have difficulty to get final concat dataframe by try except for pymssql
for i in Com_Tables:
try:
frames = [pd.read_sql(f"SELECT * FROM {i}", conn).assign(ComId_Code = i)]
except:
pass
finally:
dfxilnex = pd.concat(frames)
i am expecting output to concat all data from each item in list however the result…
ZulH
- 13
- 3
1
vote
0 answers
Pandas read_sql_query function not working
I have configured the database connection and sql query in a pandas read sql query function, but when the function is running its throwing an error.
def fetchDbTable(candidate_id):
start_time = time.time()
# candidate_id = 793
schema =…
vinoth kumar
- 101
- 2
- 11
1
vote
1 answer
Why do I have None ('NoneType' object is not iterable) when I expected a list?
I am retrieving a list of table names using pandas.read_sql and then trying to use a "for" loop to drop tables from the retrieved list. However, I am getting a 'NoneType' object is not iterable error (although the list is not…
SergiuSRG
- 13
- 5
1
vote
0 answers
Is there a maximum limit to the size of the query string for the pandas.read_sql() function?
I have a Python script that takes a list of names from the user, creates an SQL query string, and queries the database and puts the data into a dataframe using pandas.read_sql() method. However, I have noticed that if the list of people I want to…
user17193038
1
vote
1 answer
Pandas read_sql_query turning float number to int
I'm trying to extract info from an SQL database to Python. Two columns of the database are numbers, primarily in float format. My problem arises with numbers with more than 6 digits, read_sql_query reads them as int so the decimals do not appear in…
pythonrookie
- 27
- 6
1
vote
0 answers
Add confguration for type casting in pyodbc driver
I am trying to read a stored procedure by using pandas.read_sql method in Python by using the pyodbc driver.
The issue I have is when Pandas reads data, it converts int columns to float which have Null values in it.
Is there a way to fill those null…
ALTAF HUSSAIN
- 205
- 2
- 13
1
vote
1 answer
pandas `read_sql_query` - read `double` datatype in MySQL database to `Decimal`
I'm trying to use read_sql_query() to read a query from MySQL database, one of the field in the database, its type is double(24, 8), I want to use dtype= parameter to have full control of the datatypes and read it to decimal, but seems like pandas…
wawawa
- 2,835
- 6
- 44
- 105
1
vote
0 answers
Pandas `read_sql_query` with decimal data type
I'm trying to use read_sql_query() to read a query from MySQL database, one of the field in the database, its type is decimal(28,5), I want to use dtype= parameter to have full control of the datatypes, but seems like pandas can't recognise Decimal…
wawawa
- 2,835
- 6
- 44
- 105
1
vote
0 answers
How to resolve if `pandas.read.sql()` converts `bigint` to `float64` automatically?
I have a field ID, in MySQL Workbench it's Bigint(20) type, and if I query in the workbench, it looks like this:
ID
1111111111
2222222222
3333333333
...
If I use pandas.read_sql() then print out this column, it looks like…
wawawa
- 2,835
- 6
- 44
- 105