-1

I'm writing a function that executes data validation using great expectations library. The great expectation library for spark data requires to write their expectation programmatically. So im trying to execute all the functions in a for loop using a parsed yaml.

      validator, context = start_spark_df(data, table)
        for exp in yaml[table]:
            validator.{exp}

The yaml consists of

name_of_the_table:
  expect_column_values_to_not_be_null: customer
  expect_column_values_to_be_of_type: customer, int

So ideally i would like to be able to execute

validator.expect_column_values_to_not_be_null(customer)
validator.expect_column_values_to_be_of_type(customer, int)

in one for loop.

Is there any way to do it? I couldn't find anything similar to this problem on stack overflow. Thanks a bunch in advance

General Grievance
  • 4,555
  • 31
  • 31
  • 45
Jakub Strawa
  • 61
  • 8
  • Does [this](https://stackoverflow.com/questions/3061/calling-a-function-of-a-module-by-using-its-name-a-string) help with your problem? – Ted Nguyen Feb 27 '23 at 10:16
  • Does this answer your question? [Calling a function of a module by using its name (a string)](https://stackoverflow.com/questions/3061/calling-a-function-of-a-module-by-using-its-name-a-string) – Pranav Hosangadi Feb 27 '23 at 10:35

1 Answers1

0

Something like this?

validator, context = start_spark_df(data, table)

for k,v in yaml[table].items():
 if type(args) is list:
  getattr(validator, k)(*v)
 else:
  getattr(validator, k)(v)

You can use getattr to get the function by its name, in your specific case it would be the yaml key. To pass arguments instead, you must use the spread operator in case there are more than one.

Marco Riccetti
  • 84
  • 4