4

Hi, community,
  I want to publish(save) the delta live table(DLT) into a different catalog database.
  The following image target field only asks for the database name, not for the catalog. enter image description here

I've referred to documentation but couldn't find anything helpful.

Shahrukh lodhi
  • 360
  • 2
  • 13
  • Does this answer your question? [How to direct DLT target table to a Unity Catalog Metastore](https://stackoverflow.com/questions/73339731/how-to-direct-dlt-target-table-to-a-unity-catalog-metastore) – dduque Jan 19 '23 at 09:24

1 Answers1

3

Unity Catalog isn't supported in Delta Live Tables yet - as I remember, it's planned to be released really soon.

Right now, there is a workaround - you can push data into a location on S3 that then could be added as a table in Unity Catalog external location.

P.S. It's not recommended to use default "Storage location" - it won't work with Unity Catalog as it's on DBFS Root. It's better to push it to a S3.

Alex Ott
  • 80,552
  • 8
  • 87
  • 132
  • Are you able to create persistent views on external tables to unity catalog? I know you aren't able to from the hive_metastore – Marc Dec 01 '22 at 17:54
  • you should be able - I don't remember such limitation, but I don't remember out of the head – Alex Ott Dec 01 '22 at 18:29
  • also how do you then import the external dlt tables to UC? and will automatically grab the updates – Marc Dec 01 '22 at 19:45
  • The same way as other tables - create table … location …, just in UC external location – Alex Ott Dec 01 '22 at 20:00
  • 1
    I see, unfortunately I don't think it will work well for our use case because it only reads the table once... so after each update we will have to reread the table – Marc Dec 01 '22 at 21:20
  • You need to check - content will be always updated – Alex Ott Dec 02 '22 at 06:25
  • This suggests otherwise, no? https://docs.databricks.com/data-governance/unity-catalog/create-tables.html#insert-records-from-a-path-into-an-existing-table – Marc Dec 02 '22 at 18:35
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/250094/discussion-between-marc-and-alex-ott). – Marc Dec 02 '22 at 18:58
  • `COPY INTO` is used for inserting completely new data. Data written into table from "outside" are automatically visible even with the Hive Metastore - that is one of the pain points that Delta format solves compared to Parquet – Alex Ott Dec 02 '22 at 19:35