1

Implementing pipeline components in Neuraxle, i wonder if it is possible and/or advisable to have default values for hyperparameters. Looking at code and documentation, my guess is that it is not supported, but i cannot find any mention of it in the docs. I notice here that hyperparameters are set before the setup phase, which makes me suspect setting defaults in code is not "possible".

It would be nice with default values, as it would allow much more hyperparameter options without explicitly defining them when training. It would also allow adding hyperparameters without breaking existing training code. A downside with defaults is increased complexity and perhaps issues with reproducibility if defaults change.

Any insight here would be appreciated.

Joel Ödlund
  • 15
  • 4

1 Answers1

1

If I understand your question well, it is entirely possible to have default value for a hyperparameter. You can do so using by using your step class constructor function. To do so, your parameter simply needs to have a corresponding FixedHyperparameter instance entry in the hyperparameter space.

e.g.

class MyStep(BaseStep):
    def __init__(self, default_hyperparam_value):
        BaseStep.__init__(self, hyperparams = {"my_hyperparam_name":default_hyperparam_value},
                          hyperparams_space={"my_hyperparam_name":FixedHyperparameter(default_hyperparam_value)})

Alternatively, you could exclude it entirely from the hyperparameter dictionaries and simply set it as a step attribute. They are, of course, many other way of achieving similar behaviour.

Let me know if I've misunderstood your question, I'll be glad to provide any further needed insight :)

Vincent Antaki
  • 61
  • 3
  • Thank you for your answer. Using the constructor was an option i had not thought about, it works. Using step attribute does not work so well, as it will not be part of any optimization. I will try your solution. – Joel Ödlund May 03 '21 at 14:46
  • Glad I could help. If that is not too much to ask you, and you agree with it, I'd appreciate if you could flag my answer as accepted as it would allow me to make a better usage of my stackoverflow account. Cheers! – Vincent Antaki May 04 '21 at 13:04