mlr3verse机器学习

R语言
机器学习
作者

不止BI

发布于

2024年4月4日

修改于

2024年6月11日

mlr3 是一个用于机器学习的R语言包。它是mlr包的下一代版本,旨在提供更强大、更灵活的机器学习功能。mlr3 提供了一种模块化的框架,使用户可以轻松地执行各种机器学习任务。

mlr3verse 是一组用于机器学习的R语言包的集合,旨在扩展mlr3框架的功能。这个集合包括了一系列用于数据预处理、特征选择、模型调优和评估的包,为用户提供了更多的工具和功能来进行机器学习任务。mlr3verse 的包括 mlr3mlr3learnersmlr3pipelinesmlr3tuningmlr3viz 等,每个包都有其特定的功能和用途,用户可以根据自己的需求选择适合的包来完成各种机器学习任务。

创建机器学习任务

mlr3 中需要将数据封装为 task 类,然后再进行机器学习的相关操作

代码
library(mlr3verse)
tsk_penguins <- as_task_classif(palmerpenguins::penguins,
  target = "species",
  id = "penguins"
)

tsk_penguins
<TaskClassif:penguins> (344 x 8)
* Target: species
* Properties: multiclass
* Features (7):
  - int (3): body_mass_g, flipper_length_mm, year
  - dbl (2): bill_depth_mm, bill_length_mm
  - fct (2): island, sex

常用task方法

代码
# 查看列结构
tsk_penguins$col_info
Key: <id>
                  id    type                  levels  label fix_factor_levels
              <char>  <char>                  <list> <char>            <lgcl>
1:          ..row_id integer                  [NULL]   <NA>             FALSE
2:     bill_depth_mm numeric                  [NULL]   <NA>             FALSE
3:    bill_length_mm numeric                  [NULL]   <NA>             FALSE
4:       body_mass_g integer                  [NULL]   <NA>             FALSE
5: flipper_length_mm integer                  [NULL]   <NA>             FALSE
6:            island  factor  Biscoe,Dream,Torgersen   <NA>             FALSE
7:               sex  factor             female,male   <NA>             FALSE
8:           species  factor Adelie,Chinstrap,Gentoo   <NA>             FALSE
9:              year integer                  [NULL]   <NA>             FALSE
代码
# 查看数据
tsk_penguins$data(1:5)
   species bill_depth_mm bill_length_mm body_mass_g flipper_length_mm    island
    <fctr>         <num>          <num>       <int>             <int>    <fctr>
1:  Adelie          18.7           39.1        3750               181 Torgersen
2:  Adelie          17.4           39.5        3800               186 Torgersen
3:  Adelie          18.0           40.3        3250               195 Torgersen
4:  Adelie            NA             NA          NA                NA Torgersen
5:  Adelie          19.3           36.7        3450               193 Torgersen
      sex  year
   <fctr> <int>
1:   male  2007
2: female  2007
3: female  2007
4:   <NA>  2007
5: female  2007
代码
# 查看缺失值
tsk_penguins$missings()
          species     bill_depth_mm    bill_length_mm       body_mass_g 
                0                 2                 2                 2 
flipper_length_mm            island               sex              year 
                2                 0                11                 0 
代码
tsk_penguins$formula()
species ~ .
NULL

数据预处理

mlr3的数据预处理是通过mlr3pipelines实现的。mlr3pipelines中的数据预处理的所有步骤都采用了字符来表示,比较难记忆,可以在官网查看说明或通过mlr_pipeops来查看可用操作及说明

代码
library(DT)
datatable(as.data.table(mlr_pipeops), options = list(
  language = list(url = "https://cdn.datatables.net/plug-ins/1.10.11/i18n/Chinese.json"),
  pageLength = 5
))

从上表可以看到,mlr3verse 提供了丰富的数据处理方法。使用mlr3内置的iris等任务,我们抽取部分常用的数据预处理流程进行示例

代码
task <- tsk("iris")
task
<TaskClassif:iris> (150 x 5): Iris Flowers
* Target: Species
* Properties: multiclass
* Features (4):
  - dbl (4): Petal.Length, Petal.Width, Sepal.Length, Sepal.Width

中心化和标准化

mlr3pipelines 中所有的数据预处理步骤都是通过po函数实现,每一个 po 除了自己的数据预处理参数之外,可以额外指定一个id作为该步骤的唯一名称,方便之后定位该步骤。

affect_columns参数可以控制该步骤影响的列,可以通过selector_*族函数来快捷的选择列

  1. selector_all():选择所有变量。

  2. selector_none():不选择任何变量。

  3. selector_type(types):按照变量类型选择,比如字符型、数值型等。

  4. selector_grep(pattern, ignore.case = FALSE, perl = FALSE, fixed = FALSE):使用正则表达式选择变量。

  5. selector_name(feature_names, assert_present = FALSE):按照变量名选择。

  6. selector_invert(selector):反选变量,即删除指定的变量。

  7. selector_intersect(selector_x, selector_y):选择两个选择器的交集。

  8. selector_union(selector_x, selector_y):选择两个选择器的并集。

  9. selector_setdiff(selector_x, selector_y):选择两个选择器的差集。

  10. selector_missing():选择存在缺失值的变量。

  11. selector_cardinality_greater_than(min_cardinality):选择分类特征基数(唯一值的数量)大于某个值的变量。

代码
# 选择预处理步骤
pos <- po(
  "scale",
  center = T, # 中心化
  scale = F, # 标准化
  affect_columns = selector_name(c("Petal.Length", "Petal.Width", "Sepal.Length")), # 作用的变量
  id = "scale_iris" # 为该步骤命名
)
pos
PipeOp: <scale_iris> (not trained)
values: <center=TRUE, scale=FALSE, robust=FALSE, affect_columns=<Selector>>
Input channels <name [train type, predict type]>:
  input [Task,Task]
Output channels <name [train type, predict type]>:
  output [Task,Task]
代码
# 执行并提取处理后数据
pos$train(list(task))[[1]]$data()
       Species Petal.Length Petal.Width Sepal.Length Sepal.Width
        <fctr>        <num>       <num>        <num>       <num>
  1:    setosa       -2.358  -0.9993333  -0.74333333         3.5
  2:    setosa       -2.358  -0.9993333  -0.94333333         3.0
  3:    setosa       -2.458  -0.9993333  -1.14333333         3.2
  4:    setosa       -2.258  -0.9993333  -1.24333333         3.1
  5:    setosa       -2.358  -0.9993333  -0.84333333         3.6
 ---                                                            
146: virginica        1.442   1.1006667   0.85666667         3.0
147: virginica        1.242   0.7006667   0.45666667         2.5
148: virginica        1.442   0.8006667   0.65666667         3.0
149: virginica        1.642   1.1006667   0.35666667         3.4
150: virginica        1.342   0.6006667   0.05666667         3.0

缺失值处理

  • imputelearner:算法插补

  • imputemean:均值

  • imputemedian:中位数

  • imputeconstant:常数

  • imoutehist:直方图

  • imputemode:众数插补

  • imputesample:随机

均值插补

代码
task <- tsk("pima")
task$missings()
diabetes      age  glucose  insulin     mass pedigree pregnant pressure 
       0        0        5      374       11        0        0       35 
 triceps 
     227 
代码
# 决策树插补
po <- po("imputemean")
new_task <- po$train(list(task = task))[[1]]
new_task$missings()
diabetes      age pedigree pregnant  glucose  insulin     mass pressure 
       0        0        0        0        0        0        0        0 
 triceps 
       0 

算法插补

代码
task <- tsk("pima")
task$missings()
diabetes      age  glucose  insulin     mass pedigree pregnant pressure 
       0        0        5      374       11        0        0       35 
 triceps 
     227 
代码
# 决策树插补
po <- po("imputelearner", lrn("regr.rpart"))
new_task <- po$train(list(task = task))[[1]]
new_task$missings()
diabetes      age pedigree pregnant  glucose  insulin     mass pressure 
       0        0        0        0        0        0        0        0 
 triceps 
       0 

变量选择

代码
# task = mlr3::tsk("mtcars")
# filter = flt("find_correlation")
# filter$calculate(task)
# as.data.table(filter)
library(dplyr)
task <- tsk("mtcars")
pos <-
  # 去除高度相关的列
  po("filter",
    filter = mlr3filters::flt("find_correlation"),
    filter.cutoff = 0.1
  ) %>>%
  # 去掉常数
  po("removeconstants") %>>%
  # 去掉方差较小变量
  po("filter",
    filter = mlr3filters::flt("variance"),
    filter.frac = 0.5
  )
pos$train(task)[[1]]$data()
      mpg  carb  disp    hp  qsec
    <num> <num> <num> <num> <num>
 1:  21.0     4 160.0   110 16.46
 2:  21.0     4 160.0   110 17.02
 3:  22.8     1 108.0    93 18.61
 4:  21.4     1 258.0   110 19.44
 5:  18.7     2 360.0   175 17.02
 6:  18.1     1 225.0   105 20.22
 7:  14.3     4 360.0   245 15.84
 8:  24.4     2 146.7    62 20.00
 9:  22.8     2 140.8    95 22.90
10:  19.2     4 167.6   123 18.30
11:  17.8     4 167.6   123 18.90
12:  16.4     3 275.8   180 17.40
13:  17.3     3 275.8   180 17.60
14:  15.2     3 275.8   180 18.00
15:  10.4     4 472.0   205 17.98
16:  10.4     4 460.0   215 17.82
17:  14.7     4 440.0   230 17.42
18:  32.4     1  78.7    66 19.47
19:  30.4     2  75.7    52 18.52
20:  33.9     1  71.1    65 19.90
21:  21.5     1 120.1    97 20.01
22:  15.5     2 318.0   150 16.87
23:  15.2     2 304.0   150 17.30
24:  13.3     4 350.0   245 15.41
25:  19.2     2 400.0   175 17.05
26:  27.3     1  79.0    66 18.90
27:  26.0     2 120.3    91 16.70
28:  30.4     2  95.1   113 16.90
29:  15.8     4 351.0   264 14.50
30:  19.7     6 145.0   175 15.50
31:  15.0     8 301.0   335 14.60
32:  21.4     2 121.0   109 18.60
      mpg  carb  disp    hp  qsec

数据编码

独热编码

代码
data <- data.table::data.table(x = factor(letters[1:3]), y = factor(letters[1:3]))
task <- as_task_classif(data, target = "y")

poe <- po("encode", method = "one-hot")

# 默认 "one-hot"
poe$train(list(task))[[1]]$data()
        y   x.a   x.b   x.c
   <fctr> <num> <num> <num>
1:      a     1     0     0
2:      b     0     1     0
3:      c     0     0     1

treatment编码:

代码
poe$param_set$values$method <- "treatment"
poe$train(list(task))[[1]]$data()
        y   x.b   x.c
   <fctr> <num> <num>
1:      a     0     0
2:      b     1     0
3:      c     0     1

其他

不平衡数据

  • ratio:和参考类别相比的倍数;

  • reference:设置参考类别;

  • adjust:选择过采样还是欠采样

  • shuffle:是否对结果打乱顺序,默认TRUE

代码
data(hacide, package = "ROSE")

table(hacide.train$cls)

  0   1 
980  20 
代码
task <- as_task_classif(hacide.train, target = "cls")
pos <- po("classbalancing",
  ratio = 1,
  reference = "major",
  adjust = "all",
  shuffle = T
)
blanced <- pos$train(list(task))[[1]]$data()
table(blanced$cls)

  0   1 
980 980 

离散化

代码
task <- tsk("mtcars")
pos <- po("quantilebin", numsplits = 10, affect_columns = selector_name(c("disp", "hp")))

pos$train(list(task))[[1]]$data() %>% head()
     mpg       disp         hp    am  carb   cyl  drat  gear  qsec    vs    wt
   <num>      <ord>      <ord> <num> <num> <num> <num> <num> <num> <num> <num>
1:  21.0  (142,160]  (106,110]     1     4     6  3.90     4 16.46     0 2.620
2:  21.0  (142,160]  (106,110]     1     4     6  3.90     4 17.02     0 2.875
3:  22.8 (80.6,120]  (66,93.4]     1     1     4  3.85     4 18.61     1 2.320
4:  21.4  (196,276]  (106,110]     0     1     6  3.08     3 19.44     1 3.215
5:  18.7  (351,396]  (165,178]     0     2     8  3.15     3 17.02     0 3.440
6:  18.1  (196,276] (93.4,106]     0     1     6  2.76     3 20.22     1 3.460

一般流程

构建学习器

查看 mlr3 支持的算法

代码
mlr_learners
<DictionaryLearner> with 51 stored values
Keys: classif.cv_glmnet, classif.debug, classif.featureless,
  classif.glmnet, classif.kknn, classif.lda, classif.log_reg,
  classif.multinom, classif.naive_bayes, classif.nnet, classif.qda,
  classif.ranger, classif.rpart, classif.svm, classif.xgboost,
  clust.agnes, clust.ap, clust.bico, clust.birch, clust.cmeans,
  clust.cobweb, clust.dbscan, clust.dbscan_fpc, clust.diana, clust.em,
  clust.fanny, clust.featureless, clust.ff, clust.hclust,
  clust.hdbscan, clust.kkmeans, clust.kmeans, clust.MBatchKMeans,
  clust.mclust, clust.meanshift, clust.optics, clust.pam,
  clust.SimpleKMeans, clust.xmeans, regr.cv_glmnet, regr.debug,
  regr.featureless, regr.glmnet, regr.kknn, regr.km, regr.lm,
  regr.nnet, regr.ranger, regr.rpart, regr.svm, regr.xgboost

创建学习器

代码
learner <- lrn("classif.rpart")
learner
<LearnerClassifRpart:classif.rpart>: Classification Tree
* Model: -
* Parameters: xval=0
* Packages: mlr3, rpart
* Predict Types:  [response], prob
* Feature Types: logical, integer, numeric, factor, ordered
* Properties: importance, missings, multiclass, selected_features,
  twoclass, weights

查看学习器支持的超参数

代码
learner$param_set
<ParamSet(10)>
                id    class lower upper nlevels        default  value
            <char>   <char> <num> <num>   <num>         <list> <list>
 1:             cp ParamDbl     0     1     Inf           0.01 [NULL]
 2:     keep_model ParamLgl    NA    NA       2          FALSE [NULL]
 3:     maxcompete ParamInt     0   Inf     Inf              4 [NULL]
 4:       maxdepth ParamInt     1    30      30             30 [NULL]
 5:   maxsurrogate ParamInt     0   Inf     Inf              5 [NULL]
 6:      minbucket ParamInt     1   Inf     Inf <NoDefault[0]> [NULL]
 7:       minsplit ParamInt     1   Inf     Inf             20 [NULL]
 8: surrogatestyle ParamInt     0     1       2              0 [NULL]
 9:   usesurrogate ParamInt     0     2       3              2 [NULL]
10:           xval ParamInt     0   Inf     Inf             10      0

设置学习器参数

代码
learner <- lrn("classif.rpart", xval = 0, cp = 0.001)

划分数据集

stratify设置分层抽样

代码
task <- tsk("penguins") # 使用内置数据集
split <- partition(task, ratio = 0.6)

训练数据

代码
learner$train(task, row_ids = split$train)

预测

代码
prediction <- learner$predict(task, row_ids = split$test)
print(prediction)
<PredictionClassif> for 138 observations:
 row_ids     truth  response
       3    Adelie    Adelie
       4    Adelie    Adelie
       6    Adelie    Adelie
     ---       ---       ---
     331 Chinstrap    Adelie
     338 Chinstrap Chinstrap
     343 Chinstrap Chinstrap

评估模型

代码
prediction$confusion
           truth
response    Adelie Chinstrap Gentoo
  Adelie        64         3      0
  Chinstrap      4        21      1
  Gentoo         0         0     45
代码
autoplot(prediction)

代码
# 查看支持的指标
# msrs()
measures <- msrs(c("classif.acc", "classif.ce"))
prediction$score(measures)
classif.acc  classif.ce 
 0.94202899  0.05797101 

进阶用法

模型比较

使用 benchmark 可以同时进行多个任务、多个模型、多重抽样方法的模型比较

代码
design <- benchmark_grid(
  tasks = tsks(c("spam", "german_credit", "sonar")),
  learners = lrns(c("classif.ranger", "classif.rpart", "classif.featureless"), predict_type = "prob"),
  resamplings = rsmps(c("holdout", "cv"))
)
print(design)
             task             learner resampling
           <char>              <char>     <char>
 1:          spam      classif.ranger    holdout
 2:          spam      classif.ranger         cv
 3:          spam       classif.rpart    holdout
 4:          spam       classif.rpart         cv
 5:          spam classif.featureless    holdout
 6:          spam classif.featureless         cv
 7: german_credit      classif.ranger    holdout
 8: german_credit      classif.ranger         cv
 9: german_credit       classif.rpart    holdout
10: german_credit       classif.rpart         cv
11: german_credit classif.featureless    holdout
12: german_credit classif.featureless         cv
13:         sonar      classif.ranger    holdout
14:         sonar      classif.ranger         cv
15:         sonar       classif.rpart    holdout
16:         sonar       classif.rpart         cv
17:         sonar classif.featureless    holdout
18:         sonar classif.featureless         cv

Holdout和CV都是用于评估机器学习模型性能的方法,但两者之间存在一些关键差异。

Holdout

Holdout方法是将数据集划分为训练集和测试集,其中训练集用于训练模型,测试集用于评估模型性能。训练集和测试集的大小通常是固定的,例如70%的训练集和30%的测试集。

Holdout方法简单易用,但存在以下缺点:

  • 训练集和测试集的划分方式可能会影响模型性能评估结果。例如,如果训练集和测试集的分布不一致,则模型性能评估结果可能不准确。

  • Holdout方法只使用了一部分数据来训练模型,因此模型性能评估结果可能不够可靠。

CV

CV方法将数据集划分为多个子集,每个子集轮流作为训练集和测试集。这样可以使每个数据点都有机会被用作训练集和测试集,从而提高模型性能评估结果的可靠性。

常用的CV方法包括k折交叉验证和留一交叉验证。k折交叉验证将数据集划分为k个子集,每个子集轮流作为测试集,其余k-1个子集作为训练集。留一交叉验证将数据集划分为n个子集,其中每个子集包含一个数据点,每个数据点单独作为测试集,其余n-1个数据点作为训练集。

CV方法比Holdout方法更复杂,但具有以下优点:

  • CV方法可以充分利用数据,提高模型性能评估结果的可靠性。

  • CV方法可以用于选择最佳的超参数。

代码
bmr <- benchmark(design, store_models = T)
INFO  [16:55:54.000] [mlr3] Running benchmark with 99 resampling iterations
INFO  [16:55:54.071] [mlr3] Applying learner 'classif.ranger' on task 'spam' (iter 1/1)
INFO  [16:55:55.746] [mlr3] Applying learner 'classif.ranger' on task 'spam' (iter 1/10)
INFO  [16:55:57.779] [mlr3] Applying learner 'classif.ranger' on task 'spam' (iter 2/10)
INFO  [16:55:59.510] [mlr3] Applying learner 'classif.ranger' on task 'spam' (iter 3/10)
INFO  [16:56:01.265] [mlr3] Applying learner 'classif.ranger' on task 'spam' (iter 4/10)
INFO  [16:56:03.834] [mlr3] Applying learner 'classif.ranger' on task 'spam' (iter 5/10)
INFO  [16:56:06.207] [mlr3] Applying learner 'classif.ranger' on task 'spam' (iter 6/10)
INFO  [16:56:09.214] [mlr3] Applying learner 'classif.ranger' on task 'spam' (iter 7/10)
INFO  [16:56:12.729] [mlr3] Applying learner 'classif.ranger' on task 'spam' (iter 8/10)
INFO  [16:56:15.533] [mlr3] Applying learner 'classif.ranger' on task 'spam' (iter 9/10)
INFO  [16:56:18.217] [mlr3] Applying learner 'classif.ranger' on task 'spam' (iter 10/10)
INFO  [16:56:21.763] [mlr3] Applying learner 'classif.rpart' on task 'spam' (iter 1/1)
INFO  [16:56:21.850] [mlr3] Applying learner 'classif.rpart' on task 'spam' (iter 1/10)
INFO  [16:56:21.958] [mlr3] Applying learner 'classif.rpart' on task 'spam' (iter 2/10)
INFO  [16:56:22.066] [mlr3] Applying learner 'classif.rpart' on task 'spam' (iter 3/10)
INFO  [16:56:22.176] [mlr3] Applying learner 'classif.rpart' on task 'spam' (iter 4/10)
INFO  [16:56:22.282] [mlr3] Applying learner 'classif.rpart' on task 'spam' (iter 5/10)
INFO  [16:56:22.389] [mlr3] Applying learner 'classif.rpart' on task 'spam' (iter 6/10)
INFO  [16:56:22.492] [mlr3] Applying learner 'classif.rpart' on task 'spam' (iter 7/10)
INFO  [16:56:22.583] [mlr3] Applying learner 'classif.rpart' on task 'spam' (iter 8/10)
INFO  [16:56:22.674] [mlr3] Applying learner 'classif.rpart' on task 'spam' (iter 9/10)
INFO  [16:56:22.771] [mlr3] Applying learner 'classif.rpart' on task 'spam' (iter 10/10)
INFO  [16:56:22.865] [mlr3] Applying learner 'classif.featureless' on task 'spam' (iter 1/1)
INFO  [16:56:22.889] [mlr3] Applying learner 'classif.featureless' on task 'spam' (iter 1/10)
INFO  [16:56:22.911] [mlr3] Applying learner 'classif.featureless' on task 'spam' (iter 2/10)
INFO  [16:56:22.932] [mlr3] Applying learner 'classif.featureless' on task 'spam' (iter 3/10)
INFO  [16:56:22.951] [mlr3] Applying learner 'classif.featureless' on task 'spam' (iter 4/10)
INFO  [16:56:22.973] [mlr3] Applying learner 'classif.featureless' on task 'spam' (iter 5/10)
INFO  [16:56:22.995] [mlr3] Applying learner 'classif.featureless' on task 'spam' (iter 6/10)
INFO  [16:56:23.017] [mlr3] Applying learner 'classif.featureless' on task 'spam' (iter 7/10)
INFO  [16:56:23.040] [mlr3] Applying learner 'classif.featureless' on task 'spam' (iter 8/10)
INFO  [16:56:23.066] [mlr3] Applying learner 'classif.featureless' on task 'spam' (iter 9/10)
INFO  [16:56:23.090] [mlr3] Applying learner 'classif.featureless' on task 'spam' (iter 10/10)
INFO  [16:56:23.114] [mlr3] Applying learner 'classif.ranger' on task 'german_credit' (iter 1/1)
INFO  [16:56:23.404] [mlr3] Applying learner 'classif.ranger' on task 'german_credit' (iter 1/10)
INFO  [16:56:23.738] [mlr3] Applying learner 'classif.ranger' on task 'german_credit' (iter 2/10)
INFO  [16:56:24.050] [mlr3] Applying learner 'classif.ranger' on task 'german_credit' (iter 3/10)
INFO  [16:56:24.244] [mlr3] Applying learner 'classif.ranger' on task 'german_credit' (iter 4/10)
INFO  [16:56:24.447] [mlr3] Applying learner 'classif.ranger' on task 'german_credit' (iter 5/10)
INFO  [16:56:24.651] [mlr3] Applying learner 'classif.ranger' on task 'german_credit' (iter 6/10)
INFO  [16:56:24.857] [mlr3] Applying learner 'classif.ranger' on task 'german_credit' (iter 7/10)
INFO  [16:56:25.078] [mlr3] Applying learner 'classif.ranger' on task 'german_credit' (iter 8/10)
INFO  [16:56:25.279] [mlr3] Applying learner 'classif.ranger' on task 'german_credit' (iter 9/10)
INFO  [16:56:25.953] [mlr3] Applying learner 'classif.ranger' on task 'german_credit' (iter 10/10)
INFO  [16:56:26.183] [mlr3] Applying learner 'classif.rpart' on task 'german_credit' (iter 1/1)
INFO  [16:56:26.208] [mlr3] Applying learner 'classif.rpart' on task 'german_credit' (iter 1/10)
INFO  [16:56:26.237] [mlr3] Applying learner 'classif.rpart' on task 'german_credit' (iter 2/10)
INFO  [16:56:26.264] [mlr3] Applying learner 'classif.rpart' on task 'german_credit' (iter 3/10)
INFO  [16:56:26.288] [mlr3] Applying learner 'classif.rpart' on task 'german_credit' (iter 4/10)
INFO  [16:56:26.311] [mlr3] Applying learner 'classif.rpart' on task 'german_credit' (iter 5/10)
INFO  [16:56:26.338] [mlr3] Applying learner 'classif.rpart' on task 'german_credit' (iter 6/10)
INFO  [16:56:26.364] [mlr3] Applying learner 'classif.rpart' on task 'german_credit' (iter 7/10)
INFO  [16:56:26.393] [mlr3] Applying learner 'classif.rpart' on task 'german_credit' (iter 8/10)
INFO  [16:56:26.419] [mlr3] Applying learner 'classif.rpart' on task 'german_credit' (iter 9/10)
INFO  [16:56:26.445] [mlr3] Applying learner 'classif.rpart' on task 'german_credit' (iter 10/10)
INFO  [16:56:26.476] [mlr3] Applying learner 'classif.featureless' on task 'german_credit' (iter 1/1)
INFO  [16:56:26.490] [mlr3] Applying learner 'classif.featureless' on task 'german_credit' (iter 1/10)
INFO  [16:56:26.507] [mlr3] Applying learner 'classif.featureless' on task 'german_credit' (iter 2/10)
INFO  [16:56:26.522] [mlr3] Applying learner 'classif.featureless' on task 'german_credit' (iter 3/10)
INFO  [16:56:26.538] [mlr3] Applying learner 'classif.featureless' on task 'german_credit' (iter 4/10)
INFO  [16:56:26.554] [mlr3] Applying learner 'classif.featureless' on task 'german_credit' (iter 5/10)
INFO  [16:56:26.576] [mlr3] Applying learner 'classif.featureless' on task 'german_credit' (iter 6/10)
INFO  [16:56:26.594] [mlr3] Applying learner 'classif.featureless' on task 'german_credit' (iter 7/10)
INFO  [16:56:26.613] [mlr3] Applying learner 'classif.featureless' on task 'german_credit' (iter 8/10)
INFO  [16:56:26.632] [mlr3] Applying learner 'classif.featureless' on task 'german_credit' (iter 9/10)
INFO  [16:56:26.646] [mlr3] Applying learner 'classif.featureless' on task 'german_credit' (iter 10/10)
INFO  [16:56:26.661] [mlr3] Applying learner 'classif.ranger' on task 'sonar' (iter 1/1)
INFO  [16:56:26.739] [mlr3] Applying learner 'classif.ranger' on task 'sonar' (iter 1/10)
INFO  [16:56:26.825] [mlr3] Applying learner 'classif.ranger' on task 'sonar' (iter 2/10)
INFO  [16:56:26.915] [mlr3] Applying learner 'classif.ranger' on task 'sonar' (iter 3/10)
INFO  [16:56:27.006] [mlr3] Applying learner 'classif.ranger' on task 'sonar' (iter 4/10)
INFO  [16:56:27.105] [mlr3] Applying learner 'classif.ranger' on task 'sonar' (iter 5/10)
INFO  [16:56:27.204] [mlr3] Applying learner 'classif.ranger' on task 'sonar' (iter 6/10)
INFO  [16:56:27.304] [mlr3] Applying learner 'classif.ranger' on task 'sonar' (iter 7/10)
INFO  [16:56:27.397] [mlr3] Applying learner 'classif.ranger' on task 'sonar' (iter 8/10)
INFO  [16:56:27.503] [mlr3] Applying learner 'classif.ranger' on task 'sonar' (iter 9/10)
INFO  [16:56:27.592] [mlr3] Applying learner 'classif.ranger' on task 'sonar' (iter 10/10)
INFO  [16:56:27.681] [mlr3] Applying learner 'classif.rpart' on task 'sonar' (iter 1/1)
INFO  [16:56:27.703] [mlr3] Applying learner 'classif.rpart' on task 'sonar' (iter 1/10)
INFO  [16:56:27.727] [mlr3] Applying learner 'classif.rpart' on task 'sonar' (iter 2/10)
INFO  [16:56:27.750] [mlr3] Applying learner 'classif.rpart' on task 'sonar' (iter 3/10)
INFO  [16:56:27.775] [mlr3] Applying learner 'classif.rpart' on task 'sonar' (iter 4/10)
INFO  [16:56:27.802] [mlr3] Applying learner 'classif.rpart' on task 'sonar' (iter 5/10)
INFO  [16:56:27.832] [mlr3] Applying learner 'classif.rpart' on task 'sonar' (iter 6/10)
INFO  [16:56:27.864] [mlr3] Applying learner 'classif.rpart' on task 'sonar' (iter 7/10)
INFO  [16:56:27.894] [mlr3] Applying learner 'classif.rpart' on task 'sonar' (iter 8/10)
INFO  [16:56:27.922] [mlr3] Applying learner 'classif.rpart' on task 'sonar' (iter 9/10)
INFO  [16:56:27.951] [mlr3] Applying learner 'classif.rpart' on task 'sonar' (iter 10/10)
INFO  [16:56:27.981] [mlr3] Applying learner 'classif.featureless' on task 'sonar' (iter 1/1)
INFO  [16:56:27.995] [mlr3] Applying learner 'classif.featureless' on task 'sonar' (iter 1/10)
INFO  [16:56:28.013] [mlr3] Applying learner 'classif.featureless' on task 'sonar' (iter 2/10)
INFO  [16:56:28.031] [mlr3] Applying learner 'classif.featureless' on task 'sonar' (iter 3/10)
INFO  [16:56:28.047] [mlr3] Applying learner 'classif.featureless' on task 'sonar' (iter 4/10)
INFO  [16:56:28.066] [mlr3] Applying learner 'classif.featureless' on task 'sonar' (iter 5/10)
INFO  [16:56:28.083] [mlr3] Applying learner 'classif.featureless' on task 'sonar' (iter 6/10)
INFO  [16:56:28.098] [mlr3] Applying learner 'classif.featureless' on task 'sonar' (iter 7/10)
INFO  [16:56:28.112] [mlr3] Applying learner 'classif.featureless' on task 'sonar' (iter 8/10)
INFO  [16:56:28.128] [mlr3] Applying learner 'classif.featureless' on task 'sonar' (iter 9/10)
INFO  [16:56:28.146] [mlr3] Applying learner 'classif.featureless' on task 'sonar' (iter 10/10)
INFO  [16:56:28.188] [mlr3] Finished benchmark
代码
measures <- msrs(c("classif.acc", "classif.mcc"))

bmr$aggregate(measures)
       nr       task_id          learner_id resampling_id iters classif.acc
    <int>        <char>              <char>        <char> <int>       <num>
 1:     1          spam      classif.ranger       holdout     1   0.9445893
 2:     2          spam      classif.ranger            cv    10   0.9511002
 3:     3          spam       classif.rpart       holdout     1   0.8943937
 4:     4          spam       classif.rpart            cv    10   0.8950240
 5:     5          spam classif.featureless       holdout     1   0.6043025
 6:     6          spam classif.featureless            cv    10   0.6059559
 7:     7 german_credit      classif.ranger       holdout     1   0.7777778
 8:     8 german_credit      classif.ranger            cv    10   0.7630000
 9:     9 german_credit       classif.rpart       holdout     1   0.7417417
10:    10 german_credit       classif.rpart            cv    10   0.7430000
11:    11 german_credit classif.featureless       holdout     1   0.7267267
12:    12 german_credit classif.featureless            cv    10   0.7000000
13:    13         sonar      classif.ranger       holdout     1   0.8550725
14:    14         sonar      classif.ranger            cv    10   0.8350000
15:    15         sonar       classif.rpart       holdout     1   0.7391304
16:    16         sonar       classif.rpart            cv    10   0.7061905
17:    17         sonar classif.featureless       holdout     1   0.3913043
18:    18         sonar classif.featureless            cv    10   0.5330952
    classif.mcc
          <num>
 1:   0.8839099
 2:   0.8974370
 3:   0.7782326
 4:   0.7788838
 5:   0.0000000
 6:   0.0000000
 7:   0.3920031
 8:   0.3817242
 9:   0.2930620
10:   0.3407961
11:   0.0000000
12:   0.0000000
13:   0.6930077
14:   0.6792771
15:   0.4603609
16:   0.4064497
17:   0.0000000
18:   0.0000000
Hidden columns: resample_result
代码
library(ggplot2)
autoplot(bmr)

超参数调优

机器学习模型在实际应用中,往往会遇到性能不佳的问题。机器学习的模型都有默认的超参数,但默认的超参数并不一定最适合你的模型,在这种情况下,就需要进行超参数调优。

mlr3包含自动调参的策略,自动调参需要指定以下信息:

  • 搜索空间:指模型超参数取值的范围。

  • 优化算法:指用于搜索最优解的算法。

  • 评估方法:指用于评估模型性能的方法。

  • 评价指标:指用于衡量模型性能的指标

调优示例

代码
library(mlr3verse)
task <- tsk("pima")
learner <- lrn("classif.rpart")
# 查看算法支持的超参数
learner$param_set
<ParamSet(10)>
                id    class lower upper nlevels        default  value
            <char>   <char> <num> <num>   <num>         <list> <list>
 1:             cp ParamDbl     0     1     Inf           0.01 [NULL]
 2:     keep_model ParamLgl    NA    NA       2          FALSE [NULL]
 3:     maxcompete ParamInt     0   Inf     Inf              4 [NULL]
 4:       maxdepth ParamInt     1    30      30             30 [NULL]
 5:   maxsurrogate ParamInt     0   Inf     Inf              5 [NULL]
 6:      minbucket ParamInt     1   Inf     Inf <NoDefault[0]> [NULL]
 7:       minsplit ParamInt     1   Inf     Inf             20 [NULL]
 8: surrogatestyle ParamInt     0     1       2              0 [NULL]
 9:   usesurrogate ParamInt     0     2       3              2 [NULL]
10:           xval ParamInt     0   Inf     Inf             10      0

设置调参空间

代码
search_space <- ps(
  cp = p_dbl(lower = 0.001, upper = 0.1), # 复杂度参数
  minsplit = p_int(lower = 1, upper = 10)
)
search_space
<ParamSet(2)>
         id    class lower upper nlevels        default  value
     <char>   <char> <num> <num>   <num>         <list> <list>
1:       cp ParamDbl 0.001   0.1     Inf <NoDefault[0]> [NULL]
2: minsplit ParamInt 1.000  10.0      10 <NoDefault[0]> [NULL]

设置重抽样方法和性能指标

代码
cv <- rsmp("cv")
measures <- msrs(c("classif.ce", "time_train", "classif.acc"))

代码
library(mlr3tuning)
# 设置终止条件为10轮后停止,通过mlr_terminators可以查看支持的其他终止条件,比如run_time可以时长

instance <- tune(
  tuner = tnr("grid_search", resolution = 5, batch_size = 2),
  task = task,
  learner = learner,
  resampling = cv,
  measure = measures,
  search_space = search_space,
  term_evals = 10
)
INFO  [16:56:30.873] [bbotk] Starting to optimize 2 parameter(s) with '<OptimizerBatchGridSearch>' and '<TerminatorEvals> [n_evals=10, k=0]'
INFO  [16:56:30.891] [bbotk] Evaluating 2 configuration(s)
INFO  [16:56:30.898] [mlr3] Running benchmark with 20 resampling iterations
INFO  [16:56:30.905] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 1/10)
INFO  [16:56:30.929] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 2/10)
INFO  [16:56:30.958] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 3/10)
INFO  [16:56:30.980] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 4/10)
INFO  [16:56:31.006] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 5/10)
INFO  [16:56:31.033] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 6/10)
INFO  [16:56:31.061] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 7/10)
INFO  [16:56:31.090] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 8/10)
INFO  [16:56:31.117] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 9/10)
INFO  [16:56:31.147] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 10/10)
INFO  [16:56:31.173] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 1/10)
INFO  [16:56:31.196] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 2/10)
INFO  [16:56:31.220] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 3/10)
INFO  [16:56:31.242] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 4/10)
INFO  [16:56:31.264] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 5/10)
INFO  [16:56:31.286] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 6/10)
INFO  [16:56:31.306] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 7/10)
INFO  [16:56:31.325] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 8/10)
INFO  [16:56:31.344] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 9/10)
INFO  [16:56:31.364] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 10/10)
INFO  [16:56:31.385] [mlr3] Finished benchmark
INFO  [16:56:31.549] [bbotk] Result of batch 1:
INFO  [16:56:31.552] [bbotk]      cp minsplit classif.ce time_train classif.acc warnings errors
INFO  [16:56:31.552] [bbotk]  0.0010        3  0.3006835      0.006   0.6993165        0      0
INFO  [16:56:31.552] [bbotk]  0.0505        8  0.2408578      0.005   0.7591422        0      0
INFO  [16:56:31.552] [bbotk]  runtime_learners                                uhash
INFO  [16:56:31.552] [bbotk]              0.08 021bc8da-3580-4b5c-bfb1-3488b43f7b05
INFO  [16:56:31.552] [bbotk]              0.08 d3483c94-59e0-497c-9608-d82e2d1d3e6f
INFO  [16:56:31.556] [bbotk] Evaluating 2 configuration(s)
INFO  [16:56:31.564] [mlr3] Running benchmark with 20 resampling iterations
INFO  [16:56:31.586] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 1/10)
INFO  [16:56:31.625] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 2/10)
INFO  [16:56:31.650] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 3/10)
INFO  [16:56:31.679] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 4/10)
INFO  [16:56:31.709] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 5/10)
INFO  [16:56:31.734] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 6/10)
INFO  [16:56:31.752] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 7/10)
INFO  [16:56:31.771] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 8/10)
INFO  [16:56:31.790] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 9/10)
INFO  [16:56:31.812] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 10/10)
INFO  [16:56:31.839] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 1/10)
INFO  [16:56:31.864] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 2/10)
INFO  [16:56:31.890] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 3/10)
INFO  [16:56:31.919] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 4/10)
INFO  [16:56:31.949] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 5/10)
INFO  [16:56:31.972] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 6/10)
INFO  [16:56:31.995] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 7/10)
INFO  [16:56:32.016] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 8/10)
INFO  [16:56:32.039] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 9/10)
INFO  [16:56:32.062] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 10/10)
INFO  [16:56:32.086] [mlr3] Finished benchmark
INFO  [16:56:32.226] [bbotk] Result of batch 2:
INFO  [16:56:32.229] [bbotk]       cp minsplit classif.ce time_train classif.acc warnings errors
INFO  [16:56:32.229] [bbotk]  0.02575        1  0.2460526      0.006   0.7539474        0      0
INFO  [16:56:32.229] [bbotk]  0.10000        3  0.2616883      0.005   0.7383117        0      0
INFO  [16:56:32.229] [bbotk]  runtime_learners                                uhash
INFO  [16:56:32.229] [bbotk]              0.08 a2355435-5006-4b41-8120-635d920dd14a
INFO  [16:56:32.229] [bbotk]              0.08 0bf4db31-4f73-4f97-a382-98bd2741895f
INFO  [16:56:32.234] [bbotk] Evaluating 2 configuration(s)
INFO  [16:56:32.243] [mlr3] Running benchmark with 20 resampling iterations
INFO  [16:56:32.248] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 1/10)
INFO  [16:56:32.280] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 2/10)
INFO  [16:56:32.300] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 3/10)
INFO  [16:56:32.320] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 4/10)
INFO  [16:56:32.341] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 5/10)
INFO  [16:56:32.364] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 6/10)
INFO  [16:56:32.388] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 7/10)
INFO  [16:56:32.410] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 8/10)
INFO  [16:56:32.432] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 9/10)
INFO  [16:56:32.457] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 10/10)
INFO  [16:56:32.478] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 1/10)
INFO  [16:56:32.498] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 2/10)
INFO  [16:56:32.519] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 3/10)
INFO  [16:56:32.539] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 4/10)
INFO  [16:56:32.560] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 5/10)
INFO  [16:56:32.583] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 6/10)
INFO  [16:56:32.604] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 7/10)
INFO  [16:56:32.630] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 8/10)
INFO  [16:56:32.656] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 9/10)
INFO  [16:56:32.678] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 10/10)
INFO  [16:56:32.705] [mlr3] Finished benchmark
INFO  [16:56:32.845] [bbotk] Result of batch 3:
INFO  [16:56:32.847] [bbotk]       cp minsplit classif.ce time_train classif.acc warnings errors
INFO  [16:56:32.847] [bbotk]  0.07525        1  0.2460526      0.006   0.7539474        0      0
INFO  [16:56:32.847] [bbotk]  0.10000        1  0.2616883      0.004   0.7383117        0      0
INFO  [16:56:32.847] [bbotk]  runtime_learners                                uhash
INFO  [16:56:32.847] [bbotk]              0.10 f5e98795-bd80-4a58-9f5d-6ee2b461769c
INFO  [16:56:32.847] [bbotk]              0.05 5a52f66b-cacb-49ba-bf53-d2266982baed
INFO  [16:56:32.850] [bbotk] Evaluating 2 configuration(s)
INFO  [16:56:32.856] [mlr3] Running benchmark with 20 resampling iterations
INFO  [16:56:32.861] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 1/10)
INFO  [16:56:32.882] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 2/10)
INFO  [16:56:32.903] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 3/10)
INFO  [16:56:32.926] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 4/10)
INFO  [16:56:32.949] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 5/10)
INFO  [16:56:32.971] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 6/10)
INFO  [16:56:32.995] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 7/10)
INFO  [16:56:33.019] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 8/10)
INFO  [16:56:33.047] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 9/10)
INFO  [16:56:33.071] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 10/10)
INFO  [16:56:33.094] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 1/10)
INFO  [16:56:33.131] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 2/10)
INFO  [16:56:33.155] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 3/10)
INFO  [16:56:33.179] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 4/10)
INFO  [16:56:33.208] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 5/10)
INFO  [16:56:33.235] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 6/10)
INFO  [16:56:33.261] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 7/10)
INFO  [16:56:33.285] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 8/10)
INFO  [16:56:33.309] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 9/10)
INFO  [16:56:33.333] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 10/10)
INFO  [16:56:33.356] [mlr3] Finished benchmark
INFO  [16:56:33.537] [bbotk] Result of batch 4:
INFO  [16:56:33.540] [bbotk]       cp minsplit classif.ce time_train classif.acc warnings errors
INFO  [16:56:33.540] [bbotk]  0.00100       10  0.2654990      0.007   0.7345010        0      0
INFO  [16:56:33.540] [bbotk]  0.07525        6  0.2460526      0.007   0.7539474        0      0
INFO  [16:56:33.540] [bbotk]  runtime_learners                                uhash
INFO  [16:56:33.540] [bbotk]              0.11 e5b7ba78-65bb-47aa-af69-0d1f967ab722
INFO  [16:56:33.540] [bbotk]              0.12 9b6cd101-b678-4913-a8fa-7942bfb99820
INFO  [16:56:33.544] [bbotk] Evaluating 2 configuration(s)
INFO  [16:56:33.553] [mlr3] Running benchmark with 20 resampling iterations
INFO  [16:56:33.560] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 1/10)
INFO  [16:56:33.588] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 2/10)
INFO  [16:56:33.615] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 3/10)
INFO  [16:56:33.639] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 4/10)
INFO  [16:56:33.665] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 5/10)
INFO  [16:56:33.690] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 6/10)
INFO  [16:56:33.710] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 7/10)
INFO  [16:56:33.732] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 8/10)
INFO  [16:56:33.754] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 9/10)
INFO  [16:56:33.779] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 10/10)
INFO  [16:56:33.798] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 1/10)
INFO  [16:56:33.818] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 2/10)
INFO  [16:56:33.836] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 3/10)
INFO  [16:56:33.853] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 4/10)
INFO  [16:56:33.874] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 5/10)
INFO  [16:56:33.895] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 6/10)
INFO  [16:56:33.915] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 7/10)
INFO  [16:56:33.936] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 8/10)
INFO  [16:56:33.956] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 9/10)
INFO  [16:56:33.975] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 10/10)
INFO  [16:56:34.011] [mlr3] Finished benchmark
INFO  [16:56:34.149] [bbotk] Result of batch 5:
INFO  [16:56:34.151] [bbotk]      cp minsplit classif.ce time_train classif.acc warnings errors
INFO  [16:56:34.151] [bbotk]  0.0010        1  0.2994019      0.004   0.7005981        0      0
INFO  [16:56:34.151] [bbotk]  0.0505        3  0.2408578      0.006   0.7591422        0      0
INFO  [16:56:34.151] [bbotk]  runtime_learners                                uhash
INFO  [16:56:34.151] [bbotk]              0.06 8d4963c6-ee8b-4c70-9694-9e3e989bb0bf
INFO  [16:56:34.151] [bbotk]              0.07 4f947ca3-81a7-46bc-b0f9-97b6e943d4c7
INFO  [16:56:34.171] [bbotk] Finished optimizing after 10 evaluation(s)
INFO  [16:56:34.172] [bbotk] Result:
INFO  [16:56:34.175] [bbotk]      cp minsplit learner_param_vals  x_domain classif.ce time_train classif.acc
INFO  [16:56:34.175] [bbotk]   <num>    <int>             <list>    <list>      <num>      <num>       <num>
INFO  [16:56:34.175] [bbotk]  0.0505        8          <list[3]> <list[2]>  0.2408578      0.005   0.7591422
INFO  [16:56:34.175] [bbotk]  0.1000        1          <list[3]> <list[2]>  0.2616883      0.004   0.7383117
INFO  [16:56:34.175] [bbotk]  0.0010        1          <list[3]> <list[2]>  0.2994019      0.004   0.7005981
代码
# 设置搜索方法:grid_search为网格搜索,random_search 为随机搜索
# 注意这里设置的resolution = 5,表示会基于cp和minsplit设置5*5的均匀网格搜索,正常会搜索25个组合的参数,但是由于前面我们设置了最大轮次为10轮,所以10次就结束了
instance
<TuningInstanceBatchMultiCrit>
* State:  Optimized
* Objective: <ObjectiveTuningBatch:classif.rpart_on_pima>
* Search Space:
         id    class lower upper nlevels
     <char>   <char> <num> <num>   <num>
1:       cp ParamDbl 0.001   0.1     Inf
2: minsplit ParamInt 1.000  10.0      10
* Terminator: <TerminatorEvals>
* Result:
       cp minsplit classif.ce time_train classif.acc
    <num>    <int>      <num>      <num>       <num>
1: 0.0505        8  0.2408578      0.005   0.7591422
2: 0.1000        1  0.2616883      0.004   0.7383117
3: 0.0010        1  0.2994019      0.004   0.7005981
* Archive:
         cp minsplit classif.ce time_train classif.acc
      <num>    <int>      <num>      <num>       <num>
 1: 0.00100        3  0.3006835      0.006   0.6993165
 2: 0.05050        8  0.2408578      0.005   0.7591422
 3: 0.02575        1  0.2460526      0.006   0.7539474
 4: 0.10000        3  0.2616883      0.005   0.7383117
 5: 0.07525        1  0.2460526      0.006   0.7539474
 6: 0.10000        1  0.2616883      0.004   0.7383117
 7: 0.00100       10  0.2654990      0.007   0.7345010
 8: 0.07525        6  0.2460526      0.007   0.7539474
 9: 0.00100        1  0.2994019      0.004   0.7005981
10: 0.05050        3  0.2408578      0.006   0.7591422

查看调参结果

代码
instance
<TuningInstanceBatchMultiCrit>
* State:  Optimized
* Objective: <ObjectiveTuningBatch:classif.rpart_on_pima>
* Search Space:
         id    class lower upper nlevels
     <char>   <char> <num> <num>   <num>
1:       cp ParamDbl 0.001   0.1     Inf
2: minsplit ParamInt 1.000  10.0      10
* Terminator: <TerminatorEvals>
* Result:
       cp minsplit classif.ce time_train classif.acc
    <num>    <int>      <num>      <num>       <num>
1: 0.0505        8  0.2408578      0.005   0.7591422
2: 0.1000        1  0.2616883      0.004   0.7383117
3: 0.0010        1  0.2994019      0.004   0.7005981
* Archive:
         cp minsplit classif.ce time_train classif.acc
      <num>    <int>      <num>      <num>       <num>
 1: 0.00100        3  0.3006835      0.006   0.6993165
 2: 0.05050        8  0.2408578      0.005   0.7591422
 3: 0.02575        1  0.2460526      0.006   0.7539474
 4: 0.10000        3  0.2616883      0.005   0.7383117
 5: 0.07525        1  0.2460526      0.006   0.7539474
 6: 0.10000        1  0.2616883      0.004   0.7383117
 7: 0.00100       10  0.2654990      0.007   0.7345010
 8: 0.07525        6  0.2460526      0.007   0.7539474
 9: 0.00100        1  0.2994019      0.004   0.7005981
10: 0.05050        3  0.2408578      0.006   0.7591422

将训练好的参数应用于模型,重新训练数据

查看调整好的参数

代码
instance$result_learner_param_vals
[[1]]
[[1]]$xval
[1] 0

[[1]]$cp
[1] 0.0505

[[1]]$minsplit
[1] 8


[[2]]
[[2]]$xval
[1] 0

[[2]]$cp
[1] 0.1

[[2]]$minsplit
[1] 1


[[3]]
[[3]]$xval
[1] 0

[[3]]$cp
[1] 0.001

[[3]]$minsplit
[1] 1

模型性能

代码
instance$result_y
   classif.ce time_train classif.acc
        <num>      <num>       <num>
1:  0.2408578      0.005   0.7591422
2:  0.2616883      0.004   0.7383117
3:  0.2994019      0.004   0.7005981

将调优选择参数应用回模型

代码
learner$param_set$values <- instance$result_learner_param_vals[[1]]
learner$train(task)

pred <- learner$predict(task)

pred$confusion
        truth
response pos neg
     pos 150  58
     neg 118 442
代码
pred$score(msr("classif.acc"))
classif.acc 
  0.7708333 

auto_learner可以直接返回最优的那个模型

代码
task <- tsk("pima")
leanrer <- lrn("classif.rpart")
search_space <- ps(
  cp = p_dbl(0.001, 0.1),
  minsplit = p_int(1, 10)
)
cv <- rsmp("cv")
measures <- msr("classif.acc")

auto_learner <- auto_tuner(
  tuner = tnr("random_search", batch_size = 2),
  learner = learner,
  resampling = cv,
  measure = measures,
  search_space = search_space,
  term_evals = 10
)
auto_learner$train(task)
INFO  [16:56:34.529] [bbotk] Starting to optimize 2 parameter(s) with '<OptimizerBatchRandomSearch>' and '<TerminatorEvals> [n_evals=10, k=0]'
INFO  [16:56:34.558] [bbotk] Evaluating 2 configuration(s)
INFO  [16:56:34.566] [mlr3] Running benchmark with 20 resampling iterations
INFO  [16:56:34.572] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 1/10)
INFO  [16:56:34.594] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 2/10)
INFO  [16:56:34.618] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 3/10)
INFO  [16:56:34.637] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 4/10)
INFO  [16:56:34.659] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 5/10)
INFO  [16:56:34.682] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 6/10)
INFO  [16:56:34.701] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 7/10)
INFO  [16:56:34.722] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 8/10)
INFO  [16:56:34.746] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 9/10)
INFO  [16:56:34.766] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 10/10)
INFO  [16:56:34.790] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 1/10)
INFO  [16:56:34.812] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 2/10)
INFO  [16:56:34.832] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 3/10)
INFO  [16:56:34.853] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 4/10)
INFO  [16:56:34.880] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 5/10)
INFO  [16:56:34.914] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 6/10)
INFO  [16:56:34.937] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 7/10)
INFO  [16:56:34.959] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 8/10)
INFO  [16:56:34.980] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 9/10)
INFO  [16:56:35.003] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 10/10)
INFO  [16:56:35.025] [mlr3] Finished benchmark
INFO  [16:56:35.080] [bbotk] Result of batch 1:
INFO  [16:56:35.082] [bbotk]          cp minsplit classif.acc warnings errors runtime_learners
INFO  [16:56:35.082] [bbotk]  0.09917854        7   0.7357826        0      0             0.12
INFO  [16:56:35.082] [bbotk]  0.07838151        6   0.7448735        0      0             0.05
INFO  [16:56:35.082] [bbotk]                                 uhash
INFO  [16:56:35.082] [bbotk]  0e27424c-e253-426a-979c-c5060ca88b8e
INFO  [16:56:35.082] [bbotk]  005c77b1-69b0-40bd-9637-837ea8b6659c
INFO  [16:56:35.092] [bbotk] Evaluating 2 configuration(s)
INFO  [16:56:35.098] [mlr3] Running benchmark with 20 resampling iterations
INFO  [16:56:35.105] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 1/10)
INFO  [16:56:35.125] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 2/10)
INFO  [16:56:35.149] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 3/10)
INFO  [16:56:35.171] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 4/10)
INFO  [16:56:35.189] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 5/10)
INFO  [16:56:35.211] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 6/10)
INFO  [16:56:35.232] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 7/10)
INFO  [16:56:35.253] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 8/10)
INFO  [16:56:35.272] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 9/10)
INFO  [16:56:35.291] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 10/10)
INFO  [16:56:35.311] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 1/10)
INFO  [16:56:35.328] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 2/10)
INFO  [16:56:35.347] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 3/10)
INFO  [16:56:35.365] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 4/10)
INFO  [16:56:35.382] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 5/10)
INFO  [16:56:35.399] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 6/10)
INFO  [16:56:35.416] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 7/10)
INFO  [16:56:35.434] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 8/10)
INFO  [16:56:35.451] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 9/10)
INFO  [16:56:35.469] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 10/10)
INFO  [16:56:35.488] [mlr3] Finished benchmark
INFO  [16:56:35.550] [bbotk] Result of batch 2:
INFO  [16:56:35.553] [bbotk]          cp minsplit classif.acc warnings errors runtime_learners
INFO  [16:56:35.553] [bbotk]  0.07918972        1   0.7448735        0      0             0.05
INFO  [16:56:35.553] [bbotk]  0.08291213        1   0.7448735        0      0             0.06
INFO  [16:56:35.553] [bbotk]                                 uhash
INFO  [16:56:35.553] [bbotk]  aaaaf98d-c721-435d-a403-8b1468152cf5
INFO  [16:56:35.553] [bbotk]  55c17ed0-8743-4685-973e-3819f2fab616
INFO  [16:56:35.562] [bbotk] Evaluating 2 configuration(s)
INFO  [16:56:35.570] [mlr3] Running benchmark with 20 resampling iterations
INFO  [16:56:35.575] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 1/10)
INFO  [16:56:35.599] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 2/10)
INFO  [16:56:35.620] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 3/10)
INFO  [16:56:35.642] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 4/10)
INFO  [16:56:35.662] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 5/10)
INFO  [16:56:35.728] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 6/10)
INFO  [16:56:35.748] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 7/10)
INFO  [16:56:35.768] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 8/10)
INFO  [16:56:35.786] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 9/10)
INFO  [16:56:35.806] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 10/10)
INFO  [16:56:35.828] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 1/10)
INFO  [16:56:35.849] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 2/10)
INFO  [16:56:35.869] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 3/10)
INFO  [16:56:35.895] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 4/10)
INFO  [16:56:35.920] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 5/10)
INFO  [16:56:35.945] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 6/10)
INFO  [16:56:35.971] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 7/10)
INFO  [16:56:35.999] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 8/10)
INFO  [16:56:36.026] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 9/10)
INFO  [16:56:36.051] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 10/10)
INFO  [16:56:36.081] [mlr3] Finished benchmark
INFO  [16:56:36.159] [bbotk] Result of batch 3:
INFO  [16:56:36.162] [bbotk]          cp minsplit classif.acc warnings errors runtime_learners
INFO  [16:56:36.162] [bbotk]  0.08689330        6   0.7357826        0      0             0.13
INFO  [16:56:36.162] [bbotk]  0.05292542        2   0.7487697        0      0             0.06
INFO  [16:56:36.162] [bbotk]                                 uhash
INFO  [16:56:36.162] [bbotk]  e7fa3cc7-71cd-4acd-84e2-f6b58e2edc7b
INFO  [16:56:36.162] [bbotk]  e5f2a260-03b7-40e2-a4ca-7374ff1c8bba
INFO  [16:56:36.176] [bbotk] Evaluating 2 configuration(s)
INFO  [16:56:36.185] [mlr3] Running benchmark with 20 resampling iterations
INFO  [16:56:36.192] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 1/10)
INFO  [16:56:36.221] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 2/10)
INFO  [16:56:36.252] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 3/10)
INFO  [16:56:36.285] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 4/10)
INFO  [16:56:36.318] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 5/10)
INFO  [16:56:36.347] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 6/10)
INFO  [16:56:36.371] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 7/10)
INFO  [16:56:36.396] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 8/10)
INFO  [16:56:36.424] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 9/10)
INFO  [16:56:36.448] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 10/10)
INFO  [16:56:36.473] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 1/10)
INFO  [16:56:36.500] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 2/10)
INFO  [16:56:36.525] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 3/10)
INFO  [16:56:36.552] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 4/10)
INFO  [16:56:36.581] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 5/10)
INFO  [16:56:36.601] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 6/10)
INFO  [16:56:36.623] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 7/10)
INFO  [16:56:36.645] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 8/10)
INFO  [16:56:36.666] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 9/10)
INFO  [16:56:36.689] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 10/10)
INFO  [16:56:36.720] [mlr3] Finished benchmark
INFO  [16:56:36.781] [bbotk] Result of batch 4:
INFO  [16:56:36.783] [bbotk]          cp minsplit classif.acc warnings errors runtime_learners
INFO  [16:56:36.783] [bbotk]  0.05840653       10   0.7474710        0      0             0.13
INFO  [16:56:36.783] [bbotk]  0.06122640        7   0.7448735        0      0             0.07
INFO  [16:56:36.783] [bbotk]                                 uhash
INFO  [16:56:36.783] [bbotk]  b3131398-4b84-4b0d-a2d4-cb6bd2759b25
INFO  [16:56:36.783] [bbotk]  9d615792-9adc-4c26-95ac-ae04b22f8cd0
INFO  [16:56:36.796] [bbotk] Evaluating 2 configuration(s)
INFO  [16:56:36.803] [mlr3] Running benchmark with 20 resampling iterations
INFO  [16:56:36.809] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 1/10)
INFO  [16:56:36.828] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 2/10)
INFO  [16:56:36.847] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 3/10)
INFO  [16:56:36.866] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 4/10)
INFO  [16:56:36.888] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 5/10)
INFO  [16:56:36.909] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 6/10)
INFO  [16:56:36.930] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 7/10)
INFO  [16:56:36.952] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 8/10)
INFO  [16:56:36.972] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 9/10)
INFO  [16:56:36.995] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 10/10)
INFO  [16:56:37.016] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 1/10)
INFO  [16:56:37.038] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 2/10)
INFO  [16:56:37.061] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 3/10)
INFO  [16:56:37.085] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 4/10)
INFO  [16:56:37.108] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 5/10)
INFO  [16:56:37.131] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 6/10)
INFO  [16:56:37.153] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 7/10)
INFO  [16:56:37.177] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 8/10)
INFO  [16:56:37.199] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 9/10)
INFO  [16:56:37.218] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 10/10)
INFO  [16:56:37.239] [mlr3] Finished benchmark
INFO  [16:56:37.298] [bbotk] Result of batch 5:
INFO  [16:56:37.300] [bbotk]          cp minsplit classif.acc warnings errors runtime_learners
INFO  [16:56:37.300] [bbotk]  0.02437254        9   0.7539645        0      0             0.10
INFO  [16:56:37.300] [bbotk]  0.06133266        6   0.7448735        0      0             0.07
INFO  [16:56:37.300] [bbotk]                                 uhash
INFO  [16:56:37.300] [bbotk]  a000ae93-3cbb-48b2-9cef-747a4b315f40
INFO  [16:56:37.300] [bbotk]  7f291572-7081-4da4-9a0b-cc45ffe576da
INFO  [16:56:37.325] [bbotk] Finished optimizing after 10 evaluation(s)
INFO  [16:56:37.326] [bbotk] Result:
INFO  [16:56:37.328] [bbotk]          cp minsplit learner_param_vals  x_domain classif.acc
INFO  [16:56:37.328] [bbotk]       <num>    <int>             <list>    <list>       <num>
INFO  [16:56:37.328] [bbotk]  0.02437254        9          <list[3]> <list[2]>   0.7539645

auto_tuner调参的结果可以直接用于预测新数据

代码
auto_learner$predict(task)
<PredictionClassif> for 768 observations:
 row_ids truth response
       1   pos      pos
       2   neg      neg
       3   pos      neg
     ---   ---      ---
     766   neg      neg
     767   pos      neg
     768   neg      neg

特征选择

当我们拿到一份数据进行构建模型时,有很多信息对于目标变量来说,其实是无效的,如果把这些变量用于建模,只会增加噪声,降低模型的表现。

去除无效、冗余的变量,选择合适的变量的过程,就被称为特征选择

mlr3可按照一些方法,将所有预测变量计算一个分数,然后按照分数对变量进行排名和筛选

查看支持的计算方法

代码
as.data.table(mlr_filters)
Key: <key>
                  key                                                    label
               <char>                                                   <char>
 1:             anova                                             ANOVA F-Test
 2:               auc                           Area Under the ROC Curve Score
 3:            boruta                                                   Burota
 4:          carscore                   Correlation-Adjusted coRrelation Score
 5:      carsurvscore          Correlation-Adjusted coRrelation Survival Score
 6:              cmim      Minimal Conditional Mutual Information Maximization
 7:       correlation                                              Correlation
 8:              disr                       Double Input Symmetrical Relevance
 9:  find_correlation                                  Correlation-based Score
10:        importance                                         Importance Score
11:  information_gain                                         Information Gain
12:               jmi                                 Joint Mutual Information
13:              jmim            Minimal Joint Mutual Information Maximization
14:      kruskal_test                                      Kruskal-Wallis Test
15:               mim                          Mutual Information Maximization
16:              mrmr                     Minimum Redundancy Maximal Relevancy
17:             njmim Minimal Normalised Joint Mutual Information Maximization
18:       performance                                   Predictive Performance
19:       permutation                                        Permutation Score
20:            relief                                                   RELIEF
21: selected_features                               Embedded Feature Selection
22:    univariate_cox                            Univariate Cox Survival Score
23:          variance                                                 Variance
                  key                                                    label
      task_types task_properties
          <list>          <list>
 1:      classif                
 2:      classif        twoclass
 3: regr,classif                
 4:         regr                
 5:         surv                
 6: classif,regr                
 7:         regr                
 8: classif,regr                
 9:           NA                
10:      classif                
11: classif,regr                
12: classif,regr                
13: classif,regr                
14:      classif                
15: classif,regr                
16: classif,regr                
17: classif,regr                
18:      classif                
19:      classif                
20: classif,regr                
21:      classif                
22:         surv                
23:           NA                
      task_types task_properties
                                                 params
                                                 <list>
 1:                                                    
 2:                                                    
 3: pValue,mcAdj,maxRuns,doTrace,holdHistory,getImp,...
 4:                             lambda,diagonal,verbose
 5:                                  maxIPCweight,denom
 6:                                             threads
 7:                                          use,method
 8:                                             threads
 9:                                          use,method
10:                                              method
11:                     type,equal,discIntegers,threads
12:                                             threads
13:                                             threads
14:                                           na.action
15:                                             threads
16:                                             threads
17:                                             threads
18:                                              method
19:                                     standardize,nmc
20:                          neighboursCount,sampleSize
21:                                              method
22:                                                    
23:                                               na.rm
                                                 params
                                           feature_types          packages
                                                  <list>            <list>
 1:                                      integer,numeric             stats
 2:                                      integer,numeric      mlr3measures
 3:                                      integer,numeric            Boruta
 4:                              logical,integer,numeric              care
 5:                                      integer,numeric carSurv,mlr3proba
 6:                       integer,numeric,factor,ordered           praznik
 7:                                      integer,numeric             stats
 8:                       integer,numeric,factor,ordered           praznik
 9:                                      integer,numeric             stats
10: logical,integer,numeric,character,factor,ordered,...              mlr3
11:                       integer,numeric,factor,ordered     FSelectorRcpp
12:                       integer,numeric,factor,ordered           praznik
13:                       integer,numeric,factor,ordered           praznik
14:                                      integer,numeric             stats
15:                       integer,numeric,factor,ordered           praznik
16:                       integer,numeric,factor,ordered           praznik
17:                       integer,numeric,factor,ordered           praznik
18: logical,integer,numeric,character,factor,ordered,... mlr3,mlr3measures
19: logical,integer,numeric,character,factor,ordered,... mlr3,mlr3measures
20:                       integer,numeric,factor,ordered     FSelectorRcpp
21: logical,integer,numeric,character,factor,ordered,...              mlr3
22:                              integer,numeric,logical          survival
23:                                      integer,numeric             stats
                                           feature_types          packages

计算分数

代码
filter <- flt("jmim")

task <- tsk("iris")

filter$calculate(task)

filter
<FilterJMIM:jmim>: Minimal Joint Mutual Information Maximization
Task Types: classif, regr
Properties: -
Task Properties: -
Packages: praznik
Feature types: integer, numeric, factor, ordered
        feature     score
1:  Petal.Width 1.0000000
2: Sepal.Length 0.6666667
3: Petal.Length 0.3333333
4:  Sepal.Width 0.0000000

根据相关性

代码
task <- tsk("mtcars")

filter_cor <- flt("correlation")


filter_cor$param_set
<ParamSet(2)>
       id    class lower upper nlevels    default  value
   <char>   <char> <num> <num>   <num>     <list> <list>
1:    use ParamFct    NA    NA       5 everything [NULL]
2: method ParamFct    NA    NA       3    pearson [NULL]
代码
filter_cor$param_set$values <- list(method = "spearman")
filter_cor$param_set
<ParamSet(2)>
       id    class lower upper nlevels    default    value
   <char>   <char> <num> <num>   <num>     <list>   <list>
1:    use ParamFct    NA    NA       5 everything   [NULL]
2: method ParamFct    NA    NA       3    pearson spearman
代码
filter_cor$calculate(task)

filter_cor
<FilterCorrelation:correlation>: Correlation
Task Types: regr
Properties: missings
Task Properties: -
Packages: stats
Feature types: integer, numeric
    feature     score
 1:     cyl 0.9108013
 2:    disp 0.9088824
 3:      hp 0.8946646
 4:      wt 0.8864220
 5:      vs 0.7065968
 6:    carb 0.6574976
 7:    drat 0.6514555
 8:      am 0.5620057
 9:    gear 0.5427816
10:    qsec 0.4669358

计算变量重要性

代码
lrn <- lrn("classif.ranger", importance = "impurity")

task <- tsk("iris")
filter <- flt("importance", learner = lrn)
filter$calculate(task)
filter
<FilterImportance:importance>: Importance Score
Task Types: classif
Properties: missings
Task Properties: -
Packages: mlr3, mlr3learners, ranger
Feature types: logical, integer, numeric, character, factor, ordered
        feature     score
1:  Petal.Width 44.659090
2: Petal.Length 42.028596
3: Sepal.Length  9.992927
4:  Sepal.Width  2.576583

组合方法

类似超参数调优,构建不同特征变量的模型,通过模型效果来选择

代码
library(mlr3fselect)

task <- tsk("pima")
learner <- lrn("classif.rpart")
hout <- rsmp("holdout")
measure <- msr("classif.ce")

evals10 <- trm("evals", n_evals = 20)

instance <- mlr3fselect::FSelectInstanceBatchSingleCrit$new(
  task = task,
  learner = learner,
  resampling = hout,
  measure = measure,
  terminator = evals10
)
instance
<FSelectInstanceBatchSingleCrit>
* State:  Not optimized
* Objective: <ObjectiveFSelectBatch:classif.rpart_on_pima>
* Terminator: <TerminatorEvals>
代码
fselector <- fs("random_search")

lgr::get_logger("bbotk")$set_threshold("warn")

fselector$optimize(instance)
INFO  [16:56:37.886] [mlr3] Running benchmark with 10 resampling iterations
INFO  [16:56:37.892] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 1/1)
INFO  [16:56:37.910] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 1/1)
INFO  [16:56:37.926] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 1/1)
INFO  [16:56:37.942] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 1/1)
INFO  [16:56:37.959] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 1/1)
INFO  [16:56:37.977] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 1/1)
INFO  [16:56:37.992] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 1/1)
INFO  [16:56:38.006] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 1/1)
INFO  [16:56:38.027] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 1/1)
INFO  [16:56:38.049] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 1/1)
INFO  [16:56:38.066] [mlr3] Finished benchmark
INFO  [16:56:38.238] [mlr3] Running benchmark with 10 resampling iterations
INFO  [16:56:38.243] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 1/1)
INFO  [16:56:38.262] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 1/1)
INFO  [16:56:38.279] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 1/1)
INFO  [16:56:38.296] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 1/1)
INFO  [16:56:38.315] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 1/1)
INFO  [16:56:38.333] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 1/1)
INFO  [16:56:38.349] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 1/1)
INFO  [16:56:38.376] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 1/1)
INFO  [16:56:38.391] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 1/1)
INFO  [16:56:38.408] [mlr3] Applying learner 'classif.rpart' on task 'pima' (iter 1/1)
INFO  [16:56:38.424] [mlr3] Finished benchmark
      age glucose insulin   mass pedigree pregnant pressure triceps
   <lgcl>  <lgcl>  <lgcl> <lgcl>   <lgcl>   <lgcl>   <lgcl>  <lgcl>
1:   TRUE    TRUE    TRUE   TRUE     TRUE     TRUE     TRUE    TRUE
                                         features n_features classif.ce
                                           <list>      <int>      <num>
1: age,glucose,insulin,mass,pedigree,pregnant,...          8  0.2304688
代码
# 查看选中特征
instance$result_feature_set
[1] "age"      "glucose"  "insulin"  "mass"     "pedigree" "pregnant" "pressure"
[8] "triceps" 
代码
# 查看评估
instance$result_y
classif.ce 
 0.2304688 
代码
# 查看过程
as.data.table(instance$archive)
       age glucose insulin   mass pedigree pregnant pressure triceps classif.ce
    <lgcl>  <lgcl>  <lgcl> <lgcl>   <lgcl>   <lgcl>   <lgcl>  <lgcl>      <num>
 1:  FALSE    TRUE    TRUE  FALSE    FALSE    FALSE    FALSE   FALSE  0.2812500
 2:   TRUE    TRUE    TRUE   TRUE     TRUE    FALSE    FALSE   FALSE  0.2617188
 3:  FALSE   FALSE   FALSE  FALSE    FALSE     TRUE    FALSE   FALSE  0.3085938
 4:   TRUE    TRUE    TRUE   TRUE     TRUE     TRUE     TRUE    TRUE  0.2304688
 5:   TRUE   FALSE   FALSE  FALSE    FALSE    FALSE     TRUE    TRUE  0.3203125
 6:   TRUE   FALSE   FALSE  FALSE     TRUE     TRUE    FALSE   FALSE  0.3203125
 7:   TRUE   FALSE   FALSE   TRUE    FALSE    FALSE    FALSE   FALSE  0.3164062
 8:  FALSE   FALSE    TRUE  FALSE    FALSE    FALSE    FALSE   FALSE  0.3945312
 9:  FALSE    TRUE    TRUE   TRUE     TRUE     TRUE     TRUE   FALSE  0.2695312
10:   TRUE   FALSE   FALSE  FALSE     TRUE    FALSE    FALSE   FALSE  0.3125000
11:   TRUE    TRUE    TRUE   TRUE     TRUE     TRUE     TRUE    TRUE  0.2304688
12:   TRUE    TRUE    TRUE   TRUE     TRUE     TRUE     TRUE    TRUE  0.2304688
13:   TRUE   FALSE    TRUE   TRUE     TRUE    FALSE     TRUE    TRUE  0.3125000
14:  FALSE    TRUE    TRUE  FALSE     TRUE     TRUE    FALSE   FALSE  0.2656250
15:  FALSE   FALSE    TRUE  FALSE    FALSE     TRUE    FALSE   FALSE  0.3242188
16:   TRUE    TRUE    TRUE   TRUE     TRUE     TRUE     TRUE    TRUE  0.2304688
17:  FALSE   FALSE   FALSE   TRUE     TRUE     TRUE    FALSE   FALSE  0.3242188
18:   TRUE   FALSE    TRUE  FALSE     TRUE    FALSE     TRUE   FALSE  0.2617188
19:   TRUE    TRUE   FALSE  FALSE     TRUE     TRUE    FALSE   FALSE  0.2539062
20:   TRUE   FALSE   FALSE   TRUE    FALSE     TRUE    FALSE    TRUE  0.2968750
       age glucose insulin   mass pedigree pregnant pressure triceps classif.ce
    runtime_learners           timestamp batch_nr warnings errors
               <num>              <POSc>    <int>    <int>  <int>
 1:             0.00 2025-04-12 16:56:38        1        0      0
 2:             0.02 2025-04-12 16:56:38        1        0      0
 3:             0.01 2025-04-12 16:56:38        1        0      0
 4:             0.02 2025-04-12 16:56:38        1        0      0
 5:             0.01 2025-04-12 16:56:38        1        0      0
 6:             0.00 2025-04-12 16:56:38        1        0      0
 7:             0.00 2025-04-12 16:56:38        1        0      0
 8:             0.00 2025-04-12 16:56:38        1        0      0
 9:             0.01 2025-04-12 16:56:38        1        0      0
10:             0.02 2025-04-12 16:56:38        1        0      0
11:             0.00 2025-04-12 16:56:38        2        0      0
12:             0.00 2025-04-12 16:56:38        2        0      0
13:             0.00 2025-04-12 16:56:38        2        0      0
14:             0.02 2025-04-12 16:56:38        2        0      0
15:             0.01 2025-04-12 16:56:38        2        0      0
16:             0.02 2025-04-12 16:56:38        2        0      0
17:             0.02 2025-04-12 16:56:38        2        0      0
18:             0.00 2025-04-12 16:56:38        2        0      0
19:             0.01 2025-04-12 16:56:38        2        0      0
20:             0.02 2025-04-12 16:56:38        2        0      0
    runtime_learners           timestamp batch_nr warnings errors
                                           features n_features  resample_result
                                             <list>     <list>           <list>
 1:                                 glucose,insulin          2 <ResampleResult>
 2:               age,glucose,insulin,mass,pedigree          5 <ResampleResult>
 3:                                        pregnant          1 <ResampleResult>
 4:  age,glucose,insulin,mass,pedigree,pregnant,...          8 <ResampleResult>
 5:                            age,pressure,triceps          3 <ResampleResult>
 6:                           age,pedigree,pregnant          3 <ResampleResult>
 7:                                        age,mass          2 <ResampleResult>
 8:                                         insulin          1 <ResampleResult>
 9: glucose,insulin,mass,pedigree,pregnant,pressure          6 <ResampleResult>
10:                                    age,pedigree          2 <ResampleResult>
11:  age,glucose,insulin,mass,pedigree,pregnant,...          8 <ResampleResult>
12:  age,glucose,insulin,mass,pedigree,pregnant,...          8 <ResampleResult>
13:      age,insulin,mass,pedigree,pressure,triceps          6 <ResampleResult>
14:               glucose,insulin,pedigree,pregnant          4 <ResampleResult>
15:                                insulin,pregnant          2 <ResampleResult>
16:  age,glucose,insulin,mass,pedigree,pregnant,...          8 <ResampleResult>
17:                          mass,pedigree,pregnant          3 <ResampleResult>
18:                   age,insulin,pedigree,pressure          4 <ResampleResult>
19:                   age,glucose,pedigree,pregnant          4 <ResampleResult>
20:                       age,mass,pregnant,triceps          4 <ResampleResult>
                                           features n_features  resample_result
代码
# 将选中变量应用于模型
task$select(instance$result_feature_set) # 只使用选中的变量
learner$train(task)

自动选择

代码
task <- tsk("penguins")
split <- partition(task, ratio = 0.8)

afs <- auto_fselector(
  fselector = fs("random_search"),
  learner = lrn("classif.rpart"),
  resampling = rsmp("holdout"),
  measure = msr("classif.ce"),
  term_evals = 4
)


afs$train(task, row_ids = split$train)
INFO  [16:56:38.759] [mlr3] Running benchmark with 10 resampling iterations
INFO  [16:56:38.764] [mlr3] Applying learner 'classif.rpart' on task 'penguins' (iter 1/1)
INFO  [16:56:38.777] [mlr3] Applying learner 'classif.rpart' on task 'penguins' (iter 1/1)
INFO  [16:56:38.795] [mlr3] Applying learner 'classif.rpart' on task 'penguins' (iter 1/1)
INFO  [16:56:38.810] [mlr3] Applying learner 'classif.rpart' on task 'penguins' (iter 1/1)
INFO  [16:56:38.827] [mlr3] Applying learner 'classif.rpart' on task 'penguins' (iter 1/1)
INFO  [16:56:38.844] [mlr3] Applying learner 'classif.rpart' on task 'penguins' (iter 1/1)
INFO  [16:56:38.862] [mlr3] Applying learner 'classif.rpart' on task 'penguins' (iter 1/1)
INFO  [16:56:38.879] [mlr3] Applying learner 'classif.rpart' on task 'penguins' (iter 1/1)
INFO  [16:56:38.895] [mlr3] Applying learner 'classif.rpart' on task 'penguins' (iter 1/1)
INFO  [16:56:38.913] [mlr3] Applying learner 'classif.rpart' on task 'penguins' (iter 1/1)
INFO  [16:56:38.930] [mlr3] Finished benchmark
代码
afs$predict(task, row_ids = split$test)
<PredictionClassif> for 69 observations:
 row_ids     truth  response
       9    Adelie    Adelie
      11    Adelie    Adelie
      18    Adelie    Gentoo
     ---       ---       ---
     326 Chinstrap Chinstrap
     336 Chinstrap Chinstrap
     339 Chinstrap Chinstrap

pipelines

通过mlr3pipelines可以将数据预处理、建模、模型比较、集成学习结合在一起

基本用法

将数据预处理、特征选择结合

代码
graph <-
  po("imputehist", # 插补
    id = "impute_num", # 重命名步骤
    affect_columns = is.numeric # 设置处理哪些列
  ) %>>%
  po("imputeoor", id = "impute_fct", affect_columns = is.factor) %>>% # 插补因子
  po("filter", mlr3filters::flt("information_gain"),
    filter.frac = 0.95
  ) %>>%
  po("encode", method = "one-hot") %>>%
  po("learner", lrn("classif.rpart"))
graph$plot()

代码
task <- tsk("pima")
lrn_graph <- as_learner(graph)
lrn_graph$train(task)
pred <- lrn_graph$predict(task)
pred$confusion
        truth
response pos neg
     pos 207  73
     neg  61 427

分块训练

针对数据量较大时,可以将数据分块训练,最后将各块

代码
chks <- po("chunk", 4)
lrns <- ppl("greplicate", po("learner", lrn("classif.rpart")), 4)

mjv <- po("classifavg", 4)

pipeline <- chks %>>% lrns %>>% mjv
pipeline$plot(html = FALSE)

代码
task <- tsk("iris")
split <- partition(task, ratio = 0.7)

pipelrn <- as_learner(pipeline)
pipelrn$train(task, split$train)$
  predict(task, split$test)$
  score(msr("classif.acc"))
classif.acc 
  0.7333333 

装袋

代码
single_pred <- po("subsample", frac = 0.7) %>>%
  po("learner", lrn("classif.rpart")) # 建立一个模型

pred_set <- ppl("greplicate", single_pred, 10L) # 复制10次

bagging <- pred_set %>>%
  po("classifavg", innum = 10)

bagging$plot(html = FALSE)

代码
task <- tsk("iris")
split <- partition(task, ratio = 0.7)


baglrn <- as_learner(bagging)
baglrn$train(task, row_ids = split$train)
baglrn$predict(task, row_ids = split$test)$
  score(msr("classif.acc"))
classif.acc 
  0.9777778 

堆叠

代码
lrn <- lrn("classif.rpart")
lrn_0 <- po("learner_cv", lrn$clone())
lrn_0$id <- "rpart_cv"
level_0 <- gunion(list(lrn_0, po("nop")))
combined <- level_0 %>>% po("featureunion", 2)
stack <- combined %>>% po("learner", lrn$clone())
stack$plot(html = FALSE)

代码
stacklrn <- as_learner(stack)
stacklrn$train(task, split$train)
INFO  [16:56:41.865] [mlr3] Applying learner 'classif.rpart' on task 'iris' (iter 1/3)
INFO  [16:56:41.880] [mlr3] Applying learner 'classif.rpart' on task 'iris' (iter 2/3)
INFO  [16:56:41.894] [mlr3] Applying learner 'classif.rpart' on task 'iris' (iter 3/3)
代码
stacklrn$predict(task, split$test)$
  score(msr("classif.acc"))
classif.acc 
  0.9555556 

复杂案例

代码
library("magrittr")
library("mlr3learners")

rprt <- lrn("classif.rpart", predict_type = "prob")
glmn <- lrn("classif.glmnet", predict_type = "prob")

# 创建学习器
lrn_0 <- po("learner_cv", rprt, id = "rpart_cv_1")
lrn_0$param_set$values$maxdepth <- 5L
lrn_1 <- po("pca", id = "pca1") %>>% po("learner_cv", rprt, id = "rpart_cv_2")
lrn_1$param_set$values$rpart_cv_2.maxdepth <- 1L
lrn_2 <- po("pca", id = "pca2") %>>% po("learner_cv", glmn)

# 第0层
level_0 <- gunion(list(lrn_0, lrn_1, lrn_2, po("nop", id = "NOP1")))

# 第1层
level_1 <- level_0 %>>%
  po("featureunion", 4) %>>%
  po("copy", 3) %>>%
  gunion(list(
    po("learner_cv", rprt, id = "rpart_cv_l1"),
    po("learner_cv", glmn, id = "glmnt_cv_l1"),
    po("nop", id = "NOP_l1")
  ))

# 第2层
level_2 <- level_1 %>>%
  po("featureunion", 3, id = "u2") %>>%
  po("learner", rprt, id = "rpart_l2")


level_2$plot(html = FALSE)

代码
task <- tsk("iris")
lrn <- as_learner(level_2)

lrn$train(task, split$train)$
  predict(task, split$test)$
  score(msr("classif.acc"))
INFO  [16:56:42.579] [mlr3] Applying learner 'classif.rpart' on task 'iris' (iter 1/3)
INFO  [16:56:42.595] [mlr3] Applying learner 'classif.rpart' on task 'iris' (iter 2/3)
INFO  [16:56:42.613] [mlr3] Applying learner 'classif.rpart' on task 'iris' (iter 3/3)
INFO  [16:56:42.715] [mlr3] Applying learner 'classif.rpart' on task 'iris' (iter 1/3)
INFO  [16:56:42.742] [mlr3] Applying learner 'classif.rpart' on task 'iris' (iter 2/3)
INFO  [16:56:42.768] [mlr3] Applying learner 'classif.rpart' on task 'iris' (iter 3/3)
INFO  [16:56:43.059] [mlr3] Applying learner 'classif.glmnet' on task 'iris' (iter 1/3)
INFO  [16:56:43.106] [mlr3] Applying learner 'classif.glmnet' on task 'iris' (iter 2/3)
INFO  [16:56:43.142] [mlr3] Applying learner 'classif.glmnet' on task 'iris' (iter 3/3)
INFO  [16:56:43.253] [mlr3] Applying learner 'classif.rpart' on task 'iris' (iter 1/3)
INFO  [16:56:43.291] [mlr3] Applying learner 'classif.rpart' on task 'iris' (iter 2/3)
INFO  [16:56:43.335] [mlr3] Applying learner 'classif.rpart' on task 'iris' (iter 3/3)
INFO  [16:56:43.461] [mlr3] Applying learner 'classif.glmnet' on task 'iris' (iter 1/3)
INFO  [16:56:43.524] [mlr3] Applying learner 'classif.glmnet' on task 'iris' (iter 2/3)
INFO  [16:56:43.588] [mlr3] Applying learner 'classif.glmnet' on task 'iris' (iter 3/3)
classif.acc 
  0.9555556 
回到顶部