5

I am new to databricks and delta live tables. I have problem with creating delta live table in python.

enter image description here

enter image description here

enter image description here

How to create delta live table from json files in filestore?

Jelena Ajdukovic
  • 311
  • 3
  • 12

4 Answers4

2

It is a decorator, so I think you also need a function after. Meaning

@dlt.table(comment="your comment")
def get_bronze():
    df=spark.sql("""select * from myDb.MyRegisterdTable""")
    #If you wanna check logs:
    #print("bronze",df.take(5),"end")
    return df

In silver function then you can read it as:

@dlt.table
def get_silver():
    df = dlt.read("get_bronze")
    [..do_stuff...]
    return df

Also from your screenshots I am not sure, are you running all this as a pipeline or are you trying to run a Notebook? The latter does not work.

1

Sandro’s answer should solve your problem. For ingesting json files with live tables, you can check this article for some use cases https://medium.com/@chaobioz/create-delta-live-tables-dlt-dynamically-with-pyspark-e06a718199c8

Also if for production, better use auto loader as well.

Bi Chao
  • 21
  • 1
0
  1. Create a Notebook with the scripts
  2. Create a Pipeline (With your Required configs).
  3. Start the Pipeline directly, don't run the notebook Delta Tables will be created directly.

You can check this process mentioned in the documentation itself. https://docs.databricks.com/workflows/delta-live-tables/delta-live-tables-quickstart.html

Read the steps just above the requirement, I guess that would help.

-1

Could you try to install dlt before importing it?

%pip install dlt

K1209
  • 1
  • The problem ist not 'dlt' modul. It says that dlt module exist but dlt.table doesn't. – Tom Marvolo Riddle May 18 '22 at 09:11
  • As it’s currently written, your answer is unclear. Please [edit] to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers [in the help center](/help/how-to-answer). – Community May 18 '22 at 12:04
  • 1
    Warning, the above `dlt` package available via `pip install` has nothing to do with Databricks Delta Live Tables – Alain Dec 01 '22 at 18:16