0

I'm trying to connect to multiple data sources, Postgress, Snowflake and MySQL. In each of these sources I have some tables that I need to ingest in order to build my models correctly. I know I can do it for each of those individually with SQLAlchemy or JupySQL. Is there a way I can generalize my code so I instantiate the connection once, and pass the query with the connection in a loop to pull the different tables and later merge them locally?

Thanks!

A single connection:

pip install jupysql --quiet

%load_ext sql
%sql postgresql://username:password@host:port/database

%sql df << SELECT * FROM my_table;
Ido Michael
  • 109
  • 7
  • 1
    Do you want a notebook for each of these data sources or one notebook is enough? In one notebook you can create functions for each of the data source and then open different connections. So this way it should work, what's the exact challenge you have as of now? – Sergiu Mar 28 '23 at 14:46
  • I wanted to have everything within a single notebook. What I realized is that `ipython-sql` doesn't support multiple connections but `jupysql` is. I ended up having a similar solution to what you suggested. So a function with its arguments for the different DB and then getting the engine to query the different source structure and continuing with pandas! – Ido Michael Mar 29 '23 at 18:21

0 Answers0