Question

我想在基准实验后访问并绘制训练准确性和测试准确性。我使用准确性作为指标。

如果我将精度的集合设置为train.acc并创建test.acc和train.acc的列表，那么将无法绘制基准测试结果，因为数据框中有两列“ acc”类，它们是偶然相同的。但是，我将基准学习结果设置为“ both”和“ both”，即使未指定汇总，基准测试结果也包含了训练准确性。

我想到了一种解决方法，那就是从基准对象中提取train.acc并将其汇总并自己绘制。

我该怎么做？有没有更简单的方法？

谢谢！

#Learners
learner_GLM <- makeLearner(cl = "classif.glmnet")
learner_SVM <- makeLearner(cl = "classif.ksvm")
learner_PCA <- cpoPca(rank=2) %>>% learner_GLM

#Data
dataA = datasets::iris
dataB = datasets::iris

#Task
task.A = makeClassifTask(data = dataA,target = "Species" )
task.B = makeClassifTask(data = dataB,target = "Species" )
task = list(task.A, task.B )

#Resample
inner = makeResampleDesc("CV", iters = 2, predict = "both")
outer = makeResampleDesc("CV", iters = 2, predict = "both")

#Tune wrappers
##Ctrl
ctrl = makeTuneControlRandom(maxit = 3L)
#1
numeric_ps =  makeParamSet(
  makeNumericParam("s", lower = -2, upper = 2, trafo = function(x) 2^x))

learner_GLM = makeTuneWrapper(learner_GLM, resampling =inner, par.set = numeric_ps, control = ctrl, show.info = FALSE)
#2
learner_PCA <- makeTuneWrapper(learner_PCA, resampling =inner, par.set = numeric_ps, control = ctrl, show.info = FALSE)
#3
numeric_ps =  makeParamSet(
  makeNumericParam("C", lower = -2, upper = 2, trafo = function(x) 2^x),
  makeNumericParam("sigma", lower = -2, upper = 2, trafo = function(x) 2^x)
)
learner_SVM = makeTuneWrapper(learner_SVM, resampling = inner, par.set = numeric_ps, control = ctrl)

#Measures
trainaccuracy = setAggregation(acc, train.mean)
measures =  list(acc, trainaccuracy)

#BMR
learners = list(learner_GLM,learner_SVM, learner_PCA)
bmr =  benchmark(learners, task, outer, measures = measures, show.info = FALSE)

#Plot
plotBMRBoxplots(bmr, acc, style = "violin")
bmr$results$dataA$classif.glmnet.tuned$measures.train
bmr$results$dataA$classif.glmnet.tuned$measures.test

基准测试后绘制训练指标

0 个答案: