How to do file over-rides in hydra?

Question

I have a main config file, let's say config.yaml:

num_layers: 4
embedding_size: 512
learning_rate: 0.2
max_steps: 200000

I'd like to be able to override this, on the command-line, with another file, like say big_model.yaml, which I'd use conceptually like:

python my_script.py --override big_model.yaml

and big_model.yaml might look like:

num_layers: 8
embedding_size: 1024

I'd like to be able to override with an arbitrary number of such files, each one taking priority over the last. Let's say I also have fast_learn.yaml

learning_rate: 2.0

And so I'd then want to conceptually do something like:

python my_script.py --override big_model.yaml --override fast_learn.yaml

What is the easiest/most standard way to do this in hydra? (or potentially in omegaconf perhaps?)

(note that I'd like these override files to ideally just be standard yaml files, that override the earlier yaml files, ideally; though if I have to write using override DSL instead, I can do that, if that's the easiest/best/most standard way)

Answer 1

It sounds like package override might be the a good solution for you.

The documentation can be found here: https://hydra.cc/docs/next/advanced/overriding_packages

an example application can be found here: https://github.com/facebookresearch/hydra/tree/master/examples/advanced/package_overrides

using the example application as an example, you can achieve the override by doing something like

$ python simple.py db=postgresql db.pass=helloworld
db:
  driver: postgresql
  user: postgre_user
  pass: helloworld
  timeout: 10

Answer 2

Refer to the basic tutorial and read about config groups.

You can create arbitrary config groups, and select one option from each (As of Hydra 1.0, config groups options are mutually exclusive), you will need two config groups here: one can be model, with a normal, small and big model, and another can trainer, with maybe normal and fast options.

Config groups can also override things in other config groups. You can also always append to the defaults list from the command line - so you can also add additional config groups that are only used in the command line. an example for that can an 'experiment' config group. You can use it as:

$ python train.py +experiment=exp1

In such config groups that are overriding things across the entire config you should use the global package (read more about packages in the docs).

# @package _global_
num_layers: 8
embedding_size: 1024
learning_rate: 2.0