1

I am currently trying to load a large amount of data into a Databricks notebook using Teradata SQL. I was wondering if there is anything I can do to load the data into the notebook faster.

regressors_query = '''SELECT * FROM DATA_SCI_APP_OWN.HCA_DS_FIN_RX_C01 WHERE PLAN_DIM_CK = '''  + plan_num  + ''' AND  PRODUCT = '''  +  prod_id + ''''''
regressors = ps.DataFrame(teradata.get_data(regressors_query))

This is currently all I have done, but I cannot find any beneficial sources online. I am having 0 errors, the data just takes forever to load in my notebook. Any help would be amazing.

Ajeet Verma
  • 2,938
  • 3
  • 13
  • 24
az_peer
  • 11
  • 1
  • You might look at `teradataml` package, aka "Teradata Package for Python". https://docs.teradata.com – Fred Aug 01 '23 at 14:08

0 Answers0