Package website: release | dev
mlr3hyperband adds the optimization algorithms Successive Halving (Jamieson and Talwalkar 2016) and Hyperband (Li et al. 2018) to the mlr3 ecosystem. The implementation in mlr3hyperband features improved scheduling and parallelizes the evaluation of configurations. The package includes tuners for hyperparameter optimization in mlr3tuning and optimizers for black-box optimization in bbotk.
Resources
There are several sections about hyperparameter optimization in the mlr3book.
The gallery features a series of case studies on Hyperband.
- Tune the hyperparameters of XGBoost with Hyperband
- Use data subsampling and Hyperband to optimize a support vector machine.
The website features a benchmark about the performance of asynchronous successive halving.
Installation
Install the last release from CRAN:
install.packages("mlr3hyperband")Install the development version from GitHub:
pak::pak("mlr-org/mlr3hyperband")Examples
We optimize the hyperparameters of an XGBoost model on the Sonar data set. The number of boosting rounds nrounds is the fidelity parameter. We tag this parameter with "budget" in the search space.
library(mlr3hyperband)
library(mlr3learners)
learner = lrn("classif.xgboost",
nrounds = to_tune(p_int(27, 243, tags = "budget")),
eta = to_tune(1e-4, 1, logscale = TRUE),
max_depth = to_tune(1, 20),
colsample_bytree = to_tune(1e-1, 1),
colsample_bylevel = to_tune(1e-1, 1),
lambda = to_tune(1e-3, 1e3, logscale = TRUE),
alpha = to_tune(1e-3, 1e3, logscale = TRUE),
subsample = to_tune(1e-1, 1)
)We use the tune() function to run the optimization.
instance = tune(
tnr("hyperband", eta = 3),
task = tsk("pima"),
learner = learner,
resampling = rsmp("cv", folds = 3),
measures = msr("classif.ce")
)The instance contains the best-performing hyperparameter configuration.
instance$result## alpha colsample_bylevel colsample_bytree eta lambda max_depth nrounds subsample
## 1: 0.08577605 0.5570915 0.7090864 -2.191256 -0.2623144 5 81 0.4919964
## 3 variables not shown: [learner_param_vals, x_domain, classif.ce]The archive contains all evaluated hyperparameter configurations. Hyperband adds the "stage" and "braket".
as.data.table(instance$archive)[, .(stage, bracket, classif.ce, nrounds)]## stage bracket classif.ce nrounds
## 1: 0 2 0.2682292 27
## 2: 0 2 0.5117188 27
## 3: 0 2 0.5013021 27
## 4: 0 2 0.2708333 27
## 5: 0 2 0.3072917 27
## ---
## 18: 0 0 0.2395833 243
## 19: 0 0 0.2565104 243
## 20: 0 0 0.2434896 243
## 21: 2 2 0.2565104 243
## 22: 1 1 0.2539062 243We fit a final model with optimized hyperparameters to make predictions on new data.
learner$param_set$values = instance$result_learner_param_vals
learner$train(tsk("sonar"))