`TunerHyperband`

class that implements hyperband tuning (HB). HB
repeatedly calls SHA (TunerSuccessiveHalving) with different numbers of
starting configurations. A larger number of starting configurations
corresponds to a smaller budget allocated in the base stage. Each run of SHA
within HB is called a bracket. HB considers `s_max + 1`

brackets with
`s_max = floor(log(r_max / r_min, eta)`

. The most explorative bracket
`s = s_max`

constructs `s_max + 1`

stages and allocates the minimum budget
(`r_min`

) in the base stage. The minimum budget is increased in each bracket
by a factor of `eta`

and the number of starting configurations is computed so
that each bracket approximately spends the same budget. Use
`hyperband_schedule()`

to get a preview of the bracket layout.

`s` | 3 | 2 | 1 | 0 | ||||||||

`i` | `n_i` | `r_i` | `n_i` | `r_i` | `n_i` | `r_i` | `n_i` | `r_i` | ||||

0 | 8 | 1 | 6 | 2 | 4 | 4 | 8 | 4 | ||||

1 | 4 | 2 | 3 | 4 | 2 | 8 | ||||||

2 | 2 | 4 | 1 | 8 | ||||||||

3 | 1 | 8 |

`s`

is the bracket number, `i`

is stage number, `n_i`

is the number of
configurations and `r_i`

is the budget allocated to a single configuration.

The budget hyperparameter must be tagged with `"budget"`

in the search space.
The minimum budget (`r_min`

) which is allocated in the base stage of the most
explorative bracket, is set by the lower bound of the budget parameter. The
upper bound defines the maximum budget (`r_max`

) which which is allocated to
the candidates in the last stages.

## Source

Li L, Jamieson K, DeSalvo G, Rostamizadeh A, Talwalkar A (2018).
“Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization.”
*Journal of Machine Learning Research*, **18**(185), 1-52.
https://jmlr.org/papers/v18/16-558.html.

## Subsample Budget

If the learner lacks a natural budget parameter, mlr3pipelines::PipeOpSubsample can be applied to use the subsampling rate as budget parameter. The resulting mlr3pipelines::GraphLearner is fitted on small proportions of the mlr3::Task in the first stage, and on the complete task in last stage.

## Dictionary

This Optimizer can be instantiated via the dictionary
mlr_optimizers or with the associated sugar function `opt()`

:

```
$get("hyperband")
mlr_optimizersopt("hyperband")
```

## Parameters

`eta`

`numeric(1)`

With every stage, the budget is increased by a factor of`eta`

and only the best`1 / eta`

configurations are promoted to the next stage. Non-integer values are supported, but`eta`

is not allowed to be less or equal 1.`sampler`

paradox::Sampler

Object defining how the samples of the parameter space should be drawn in the base stage of each bracket. The default is uniform sampling.`repetitions`

`integer(1)`

If`1`

(default), optimization is stopped once all brackets are evaluated. Otherwise, optimization is stopped after`repetitions`

runs of hyperband. The bbotk::Terminator might stop the optimization before all repetitions are executed.

## Archive

The mlr3tuning::ArchiveTuning holds the following additional columns that are specific to the hyperband algorithm:

`bracket`

(`integer(1)`

)

The bracket index. Counts down to 0.`stage`

(`integer(1))`

The stages of each bracket. Starts counting at 0.`repetition`

(`integer(1))`

Repetition index. Start counting at 1.

## Custom Sampler

Hyperband supports custom paradox::Sampler object for initial
configurations in each bracket.
A custom sampler may look like this (the full example is given in the
*examples* section):

```
# - beta distribution with alpha = 2 and beta = 5
# - categorical distribution with custom probabilities
= SamplerJointIndep$new(list(
sampler $new(params[[2]], function(n) rbeta(n, 2, 5)),
Sampler1DRfun$new(params[[3]], prob = c(0.2, 0.3, 0.5))
Sampler1DCateg ))
```

## Progress Bars

`$optimize()`

supports progress bars via the package progressr
combined with a Terminator. Simply wrap the function in
`progressr::with_progress()`

to enable them. We recommend to use package
progress as backend; enable with `progressr::handlers("progress")`

.

## Parallelization

This hyperband implementation evaluates hyperparameter configurations of equal budget across brackets in one batch.
For example, all configurations in stage 1 of bracket 3 and stage 0 of bracket 2 in one batch.
To select a parallel backend, use the `plan()`

function of the future package.

## Logging

Hyperband uses a logger (as implemented in lgr) from package
bbotk.
Use `lgr::get_logger("bbotk")`

to access and control the logger.

## Super classes

`mlr3tuning::Tuner`

-> `mlr3tuning::TunerFromOptimizer`

-> `TunerHyperband`

## Examples

```
if(requireNamespace("xgboost")) {
library(mlr3learners)
# define hyperparameter and budget parameter
search_space = ps(
nrounds = p_int(lower = 1, upper = 16, tags = "budget"),
eta = p_dbl(lower = 0, upper = 1),
booster = p_fct(levels = c("gbtree", "gblinear", "dart"))
)
# \donttest{
# hyperparameter tuning on the pima indians diabetes data set
instance = tune(
method = "hyperband",
task = tsk("pima"),
learner = lrn("classif.xgboost", eval_metric = "logloss"),
resampling = rsmp("cv", folds = 3),
measures = msr("classif.ce"),
search_space = search_space,
term_evals = 100
)
# best performing hyperparameter configuration
instance$result
# }
}
#> Loading required namespace: xgboost
#> nrounds eta booster learner_param_vals x_domain classif.ce
#> 1: 8 0.2890959 gbtree <list[7]> <list[3]> 0.2486979
```