6

I am running pandas 0.16 and sqlalchemy 0.99. I am using the pandas.DataFrame.to_sql() method, which relies on sqlalchemy, to export dataframes to a MS SQL Server database.

My question is: is there a way to generate a text file with only the SQL statements created by the to_sql method, to understand exactly what is going on under the hood?

I tried this: How to retrieve executed SQL code from SQLAlchemy

but I get some SQL statements mxied with lots of other information, whereas I would need SQL statements only.

I'd need it because to_sql is too slow (6 minutes for 11MBs of data!) so I'd like to understand what's going on. I'm afraid the pandas developers will say it depends on sqlalchemy, and viceversa!

Thanks!

Community
  • 1
  • 1
Pythonista anonymous
  • 8,140
  • 20
  • 70
  • 112
  • 1
    You can try to run *SQL Server Profiler* and take a look at what happens directly on the database level. Alternatively, you could try to use various values for a `chunksize` argument, which might have an impact depending on the number of rows you want to return. But *pandas* basically does few queries: check if table exists, create it if needed, and the insert statement itself. – van Apr 22 '15 at 14:11
  • You don't provide a lot of info (like the code you are using)... indeed you take a look to the *Server profiler*. Also you could skip `pandas.DataFrame.to_sql()`, build the queries with `sqlalchemy` and take a look with `print(q.statement)`. But if you are inserting data there's not really much to analyze in the queries themselves, you'll see inserts. – lrnzcig Apr 23 '15 at 09:30
  • I think you got an answer here: https://groups.google.com/d/msg/sqlalchemy/7KYmVpby13U/3jMVXt7DusIJ ? If so, you can post it here as answer for others – joris Jun 25 '15 at 20:20

0 Answers0