Skip to contents

TunerHyperband class that implements hyperband tuning (HB). HB repeatedly calls SHA (TunerSuccessiveHalving) with different numbers of starting configurations. A larger number of starting configurations corresponds to a smaller budget allocated in the base stage. Each run of SHA within HB is called a bracket. HB considers s_max + 1 brackets with s_max = floor(log(r_max / r_min, eta). The most explorative bracket s = s_max constructs s_max + 1 stages and allocates the minimum budget (r_min) in the base stage. The minimum budget is increased in each bracket by a factor of eta and the number of starting configurations is computed so that each bracket approximately spends the same budget. Use hyperband_schedule() to get a preview of the bracket layout.


s is the bracket number, i is stage number, n_i is the number of configurations and r_i is the budget allocated to a single configuration.

The budget hyperparameter must be tagged with "budget" in the search space. The minimum budget (r_min) which is allocated in the base stage of the most explorative bracket, is set by the lower bound of the budget parameter. The upper bound defines the maximum budget (r_max) which which is allocated to the candidates in the last stages.


Li L, Jamieson K, DeSalvo G, Rostamizadeh A, Talwalkar A (2018). “Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization.” Journal of Machine Learning Research, 18(185), 1-52.

Subsample Budget

If the learner lacks a natural budget parameter, mlr3pipelines::PipeOpSubsample can be applied to use the subsampling rate as budget parameter. The resulting mlr3pipelines::GraphLearner is fitted on small proportions of the mlr3::Task in the first stage, and on the complete task in last stage.


This Optimizer can be instantiated via the dictionary mlr_optimizers or with the associated sugar function opt():




With every stage, the budget is increased by a factor of eta and only the best 1 / eta configurations are promoted to the next stage. Non-integer values are supported, but eta is not allowed to be less or equal 1.


Object defining how the samples of the parameter space should be drawn in the base stage of each bracket. The default is uniform sampling.


If 1 (default), optimization is stopped once all brackets are evaluated. Otherwise, optimization is stopped after repetitions runs of hyperband. The bbotk::Terminator might stop the optimization before all repetitions are executed.


The mlr3tuning::ArchiveTuning holds the following additional columns that are specific to the hyperband algorithm:

  • bracket (integer(1))
    The bracket index. Counts down to 0.

  • stage (integer(1))
    The stages of each bracket. Starts counting at 0.

  • repetition (integer(1))
    Repetition index. Start counting at 1.

Custom Sampler

Hyperband supports custom paradox::Sampler object for initial configurations in each bracket. A custom sampler may look like this (the full example is given in the examples section):

# - beta distribution with alpha = 2 and beta = 5
# - categorical distribution with custom probabilities
sampler = SamplerJointIndep$new(list(
  Sampler1DRfun$new(params[[2]], function(n) rbeta(n, 2, 5)),
  Sampler1DCateg$new(params[[3]], prob = c(0.2, 0.3, 0.5))

Progress Bars

$optimize() supports progress bars via the package progressr combined with a Terminator. Simply wrap the function in progressr::with_progress() to enable them. We recommend to use package progress as backend; enable with progressr::handlers("progress").


This hyperband implementation evaluates hyperparameter configurations of equal budget across brackets in one batch. For example, all configurations in stage 1 of bracket 3 and stage 0 of bracket 2 in one batch. To select a parallel backend, use future::plan().


Hyperband uses a logger (as implemented in lgr) from package bbotk. Use lgr::get_logger("bbotk") to access and control the logger.

Super classes

mlr3tuning::Tuner -> mlr3tuning::TunerFromOptimizer -> TunerHyperband


Inherited methods

Method new()

Creates a new instance of this R6 class.


Method clone()

The objects of this class are cloneable with this method.


TunerHyperband$clone(deep = FALSE)



Whether to make a deep clone.


if(requireNamespace("xgboost")) {

  # define hyperparameter and budget parameter
  search_space = ps(
    nrounds = p_int(lower = 1, upper = 16, tags = "budget"),
    eta = p_dbl(lower = 0, upper = 1),
    booster = p_fct(levels = c("gbtree", "gblinear", "dart"))

  # \donttest{
  # hyperparameter tuning on the pima indians diabetes data set
  instance = tune(
    method = "hyperband",
    task = tsk("pima"),
    learner = lrn("classif.xgboost", eval_metric = "logloss"),
    resampling = rsmp("cv", folds = 3),
    measures = msr("classif.ce"),
    search_space = search_space,
    term_evals = 100

  # best performing hyperparameter configuration
  # }
#> Loading required namespace: xgboost
#>    nrounds       eta booster learner_param_vals  x_domain classif.ce
#> 1:       8 0.5512812  gbtree          <list[6]> <list[3]>  0.2395833