0

I want to use be able to run through a list of config files and use %run to import variables from config files into a databricks notebook. But I cant find a method to dynamically change the file following %run.

I have tried specifying a parameter like this: config = './config.py' %run $config But it doesn't work. I cannot use dbutils.notebook.run(config) as I won't get access to the variables in my main notebook.

Can anything think of a way to do this?

Dylan Rees
  • 23
  • 1
  • 4

1 Answers1

0

Since, you have already mentioned config files, I will consider that you have the config files already available in some path and those are not Databricks notebook.

You can use python - configparser in one notebook to read the config files and specify the notebook path using %run in main notebook (or you can ignore the notebook itself by using configparser in main notebook)

Reference: How to read a config file using python

SreedeepS
  • 1
  • 4
  • Can config parser parse .py files? I wan't to keep my config in .py files as there's some complex datatypes in there. Usually this would be easy as you just >import config but you can't do that in Databricks, hence wanting to use %run. But I wan't to dynamically run through a different of config.py files depending on the use case. My best solution atm is keeping >%run Config.py static and dynamically renaming the other .py files to ./config.py, depending on which one I want to use. – Dylan Rees Jun 02 '21 at 09:21