2

I am trying to create a new data source in Dremio using REST API.

While the docs are pretty decent, they lack real examples which is where I fail.

So wanting to add a new data source I do the following REST calls:

  1. Login to get the session token
  2. Create a datasource: So I can create a folder structure or create a virtual dataset which is based on SQL query of pre-existing data. However I am failing to add a new PHYSICAL_DATASET or promote VIRTUAL_DATASET to a physical one.

I am getting HTTP 400 with a message that the entity to promote with a given path does not exist (when I can verify it does)

Anybody with any experience using Dremio's REST should be able to shed some light.

Thank you!

diginoise
  • 7,352
  • 2
  • 31
  • 39

1 Answers1

2

Can you describe your PHYSICAL_DATASET

  • format csv, parquet?
  • Source NAS?

can you add an example of your code and file here

here example for csv

  1. authenticate to get token

  2. Call POST localhost:9047/api/v3/catalog/{id}

    { "entityType": "dataset", "path": ["REST", "data.csv"], "type": "PHYSICAL_DATASET", "format": { "type": "Text", "fieldDelimiter": ",", "lineDelimiter": "\n", "quote": """, "escape": """, "skipFirstLine": false, "extractHeader": true, "trimHeader": false, "autoGenerateColumnNames": true }

    }

to get {id} localhost:9047/api/v3/catalog/by-path/SPACE/FILE =>"id": ? {id} should be URLencoded

Vikkes
  • 463
  • 4
  • 17