nested branches (and dependencies) in mlr3

Question

I'm trying to use information_gain and mrmr feature filtering, but also a combination of information_gain and mrmr feature filtering (the union of the two). I've tried creating a reprex below.

library("mlr3verse")
task <- tsk('sonar')


filters = list("nop" = po("nop"),
               "information_gain" = po("filter", flt("information_gain")),
               "mrmr" = po("filter", flt("mrmr")),
               "ig_mrmr" = po("branch", c("ig2", "mrmr2"), id = "ig_mrmr") %>>%
                 gunion(list("ig2" = po("filter", flt("information_gain")),
                             "mrmr2" = po("filter", flt("mrmr")))) %>>%
                 po("featureunion", id = "union_igmrmr"))

pipe =
  po("branch", names(filters), id = "branch1") %>>%
  gunion(unname(filters)) %>>%
  po("unbranch", names(filters), id = "unbranch1") %>>%
  po(lrn('classif.rpart'))

pipe$plot()

pipe plot

Looks good so far, and here you can see that I'm trying to combine the ig & mrmr selected features.

Next I set the parameters, which I think is correct:

ps <- ParamSet$new(list(
  ParamDbl$new("classif.rpart.cp", lower = 0, upper = 0.05),
  ParamInt$new("information_gain.filter.nfeat", lower = 20L, upper = 60L),
  ParamFct$new("information_gain.type", levels = c("infogain", "symuncert")),
  ParamInt$new("ig2.filter.nfeat", lower = 20L, upper = 60L),
  ParamFct$new("ig2.type", levels = c("infogain", "symuncert")),
  ParamInt$new("mrmr.filter.nfeat", lower = 20L, upper = 60L),
  ParamInt$new("mrmr2.filter.nfeat", lower = 20L, upper = 60L),
  ParamFct$new("branch1.selection", levels = names(filters)),
  ParamFct$new("ig_mrmr.selection", levels = c("ig2", "mrmr2"))
))

The dependencies are where I'm struggling. I can set the "nested" parameters on EITHER the outer branch or the inner branch, but I'm not sure how to trigger them on BOTH. In the example below they are set on the outer branch.

ps$add_dep("information_gain.filter.nfeat", "branch1.selection", CondEqual$new("information_gain"))
ps$add_dep("information_gain.type", "branch1.selection", CondEqual$new("information_gain"))
ps$add_dep("mrmr.filter.nfeat", "branch1.selection", CondEqual$new("mrmr"))
ps$add_dep("ig2.filter.nfeat", "branch1.selection", CondEqual$new("ig_mrmr"))
ps$add_dep("ig2.type", "branch1.selection", CondEqual$new("ig_mrmr"))
ps$add_dep("mrmr2.filter.nfeat", "branch1.selection", CondEqual$new("ig_mrmr"))

ps

glrn <- GraphLearner$new(pipe) 

glrn$predict_type <- "prob"

cv5 <- rsmp("cv", folds = 5)

task$col_roles$stratum <- task$target_names

instance <- TuningInstanceSingleCrit$new(
  task = task,
  learner = glrn,
  resampling = cv5,
  measure = msr("classif.auc"),
  search_space = ps,
  terminator = trm("evals", n_evals = 5)
)

tuner <- tnr("random_search")
tuner$optimize(instance)

Note that I don't hit an error until I try to optimize the tuner.

Error message:

Error in self$assert(xs) : 
  Assertion on 'xs' failed: Parameter 'ig2.filter.nfeat' not available. Did you mean 'branch1.selection' / 'information_gain.filter.nfeat' / 'information_gain.filter.frac'?.

based on your explanation I think you do not intend to use branch for "ig_mrmr" but rather copy. Branching is one way or the other while you want the graph to run both ways and then union the results. — missuse, Dec 12 '20 at 11:26
@missuse That sounds possible. In another segment (not included as part of the reprex) I use "featureunion" to combine the PCA and the raw data (as in cbind(raw, pca)), so I thought this would work similarly. I'm largely basing my code on an example of yours I found from January '20. Can you please provide a pseudocode snippet to help me get started? — Alexander Rajan, Dec 12 '20 at 13:13
@missuse I tried swapping in ` "ig_mrmr" = po("copy", 2) %>>%` for the branch and removing the parameter space for the branch, same error. It can't find the parameters for the ig2 and mrmr2, so I think it's something wrong with the logic in my dependencies. — Alexander Rajan, Dec 12 '20 at 13:25

missuse · Accepted Answer · 2020-12-14T11:55:26.373

From your description it sounds as if you do not intend to use a branch for c("ig2", "mrmr2"):

po("branch", c("ig2", "mrmr2"), id = "ig_mrmr") %>>%
                 gunion(list("ig2" = po("filter", flt("information_gain")),
                             "mrmr2" = po("filter", flt("mrmr")))) %>>%
                 po("featureunion", id = "union_igmrmr")

since you intend to combine the output of these two. In other words you want them both applied in the same instance of resampling.

library("mlr3verse")
task <- tsk('sonar')
filters = list("nop" = po("nop"),
               "information_gain" = po("filter", flt("information_gain")),
               "mrmr" = po("filter", flt("mrmr")),
               "ig_mrmr" = po("copy", 2) %>>%
                 gunion(list("ig2" = po("filter", flt("information_gain")),
                                       "mrmr2" = po("filter", flt("mrmr")))) %>>%
                 po("featureunion", id = "union_igmrmr"))

pipe = po("branch", names(filters), id = "branch1") %>>%
  gunion(unname(filters)) %>>%
  po("unbranch", names(filters), id = "unbranch1") %>>%
  po(lrn('classif.rpart'))

pipe$plot()

The easiest way to see the parameters you can tune is:

pipe$param_set

From this you would see some parameters you specified do not have full names. For instance:

15:   ig2.information_gain.filter.nfeat ParamInt     0   Inf                                   <NoDefault[3]>      
16:    ig2.information_gain.filter.frac ParamDbl     0     1                                   <NoDefault[3]>      
17:  ig2.information_gain.filter.cutoff ParamDbl  -Inf   Inf                                   <NoDefault[3]>      
18:           ig2.information_gain.type ParamFct    NA    NA      infogain,gainratio,symuncert       infogain      
19:          ig2.information_gain.equal ParamLgl    NA    NA                        TRUE,FALSE          FALSE      
20:   ig2.information_gain.discIntegers ParamLgl    NA    NA                        TRUE,FALSE           TRUE      
21:        ig2.information_gain.threads ParamInt     0   Inf                                                1      
22: ig2.information_gain.affect_columns ParamUty    NA    NA                                    <Selector[1]>      
23:             mrmr2.mrmr.filter.nfeat ParamInt     0   Inf                                   <NoDefault[3]>      
24:              mrmr2.mrmr.filter.frac ParamDbl     0     1                                   <NoDefault[3]>      
25:            mrmr2.mrmr.filter.cutoff ParamDbl  -Inf   Inf                                   <NoDefault[3]>      
26:                  mrmr2.mrmr.threads ParamInt     0   Inf                                                0      
27:           mrmr2.mrmr.affect_columns ParamUty    NA    NA                                    <Selector[1]>

Lets specify correct names for parameters:

ps = ParamSet$new(list(
  ParamDbl$new("classif.rpart.cp", lower = 0, upper = 0.05),
  ParamInt$new("information_gain.filter.nfeat", lower = 20L, upper = 60L),
  ParamFct$new("information_gain.type", levels = c("infogain", "symuncert")),
  ParamInt$new("ig2.information_gain.filter.nfeat", lower = 20L, upper = 60L),
  ParamFct$new("ig2.information_gain.type", levels = c("infogain", "symuncert")),
  ParamInt$new("mrmr.filter.nfeat", lower = 20L, upper = 60L),
  ParamInt$new("mrmr2.mrmr.filter.nfeat", lower = 20L, upper = 60L),
  ParamFct$new("branch1.selection", levels = names(filters))
))

ps$add_dep("information_gain.filter.nfeat", "branch1.selection", CondEqual$new("information_gain"))
ps$add_dep("information_gain.type", "branch1.selection", CondEqual$new("information_gain"))
ps$add_dep("mrmr.filter.nfeat", "branch1.selection", CondEqual$new("mrmr"))
ps$add_dep("ig2.information_gain.filter.nfeat", "branch1.selection", CondEqual$new("ig_mrmr"))
ps$add_dep("ig2.information_gain.type", "branch1.selection", CondEqual$new("ig_mrmr"))
ps$add_dep("mrmr2.mrmr.filter.nfeat", "branch1.selection", CondEqual$new("ig_mrmr"))

and now everything runs without problems:

glrn <- GraphLearner$new(pipe) 

glrn$predict_type <- "prob"

cv5 <- rsmp("cv", folds = 5)

task$col_roles$stratum <- task$target_names

instance <- TuningInstanceSingleCrit$new(
  task = task,
  learner = glrn,
  resampling = cv5,
  measure = msr("classif.auc"),
  search_space = ps,
  terminator = trm("evals", n_evals = 5)
)

tuner <- tnr("random_search")
tuner$optimize(instance)

instance$result
   classif.rpart.cp information_gain.filter.nfeat information_gain.type ig2.information_gain.filter.nfeat ig2.information_gain.type mrmr.filter.nfeat mrmr2.mrmr.filter.nfeat branch1.selection
1:       0.01956043                            NA                  <NA>                                44                 symuncert                NA                      34           ig_mrmr
   learner_param_vals  x_domain classif.auc
1:          <list[6]> <list[5]>   0.7187196

This gallery post will be useful:

https://mlr3gallery.mlr-org.com/posts/2020-04-23-pipelines-selectors-branches/

as well as others

https://mlr3gallery.mlr-org.com/

If you feel some aspect of mlr3 is not understandable and you can not find a suitable gallery post/book example you should request it.

Link to book: https://mlr3book.mlr-org.com/

nested branches (and dependencies) in mlr3

1 Answers1