I am trying to find the very best configuration of hiperparameters for my LSTM using HYDRA. The problem is that there are several houndreds of possible permutations and I want to test them all. So I am trying to use HYDRA with JobLib Launcher to do this but I just cant get it right.
My config folder contains 15 parameters and every single one has at least 3 values. My default configuration file is made like this:
defaults:
- currency: ???
- network: ???
- optimizer: ???
- lstm_layers: ???
- lstm_neurons: ???
- learning_rate: ???
- momentum: ???
- dropout: ???
- activation: ???
- loss: ???
- epochs: ???
- batch_size: ???
- window: ???
- time_period: ???
- exchange: ???
- _self_
- override hydra/launcher: joblib
hydra:
sweeper:
params:
currency: glob(*)
network: glob(*)
optimizer: glob(*)
lstm_layers: glob(*)
lstm_neurons: glob(*)
learning_rate: glob(*)
momentum: glob(*)
dropout: glob(*)
activation: glob(*)
loss: glob(*)
epochs: glob(*)
batch_size: glob(*)
window: glob(*)
time_period: glob(*)
exchange: glob(*)
After calling python hydra_framework.py --multirun The program freezes on:
/home/*****/scpxd/hydra_framework.py:23: UserWarning:
The version_base parameter is not specified.
Please specify a compatability version level, or None.
Will assume defaults for version 1.1
@hydra.main(config_path="./conf", config_name="config")
One more thing about the hardware. It is CUDA accelerated crypto miner with 5 GTX 1080ti GPU's. I know that it cannot run all of them in parallel, but I am looking for a solution where there will be 10 models learning at the same time and after one completes, the next starts its learning process. Is it achievable with HYDRA? Thank You in advance for your help.
