1

In Databricks repos, say I want to programmatically run a notebook titled 'ExampleNotebook_45', but I want only use the part of the title 'ExampleNotebook', how do I write this in the command:

dbutils.notebook.run()

Similar language that comes to mind is the LIKE operator in sql where I can write along the lines of:

select * from table where name LIKE 'ar%'

Gitted
  • 41
  • 8
  • Notebooks are hard-coded names. You'd need to somehow _find_ all existing notebook name strings (`dbutils.fs.ls`, maybe), then filter the ones you want to run (`if 'ExampleNotebook' in notebook_name: dbutils.notebook.run(notebook_name)`) – OneCricketeer Apr 05 '22 at 00:05
  • How do I find all the notebook name strings in a folder? Before filtering out the one with '_45' in them? – Gitted Apr 05 '22 at 00:12
  • Like I said, `dbutils.fs.ls` method looks useful, assuming notebooks are stored in DBFS – OneCricketeer Apr 05 '22 at 00:16
  • as mentioned, the best way is to find the names of all notebooks in the directory and then loop over them. To find the notebook name check [this](https://stackoverflow.com/questions/53523560/databricks-how-do-i-get-path-of-current-notebook) – intruderr Apr 05 '22 at 07:29

1 Answers1

3

After doing some research, this is the answer that solved the problem, getting the list of files in the folder via the workspace API

import requests
import json

ctx = json.loads(dbutils.entry_point.getDbutils().notebook().getContext.toJson()) 
my_pat = ctx['extraContext']['api_token'] 
workspace_url = 'https://'

def list_files(base_path: str):
  lst = requests.request(method='get',
    url=f"{workspace_url}/api/2.0/workspace/list",
    headers={"Authentication": f"Bearer {my_pat}"}, 
    json={"path": base_path}).json()["objects"]
  results = []
  for i in lst:
    if i["object_type"] == "DIRECTORY" or i["object_type"] == "REPO":
      results.extend(list_files(i["path"]))
    else:
      results.append(i["path"])
  
  return results
  
all_files = list_files("base_path")

for i in all_files:
  if 'ExampleNotebook' in i:
    file_name = i
  else:
    continue

This will give me the file name (and path), that is similar to the LIKE sql operator.

Gitted
  • 41
  • 8