使用pROC绘制ROC曲线失败

Question

我有一个像这样组织的数据集：

> head(crypto_data)
                 time btc_price  btc_change btc_change_label eth_price block_size difficulty estimated_btc_sent estimated_transaction_volume_usd
1 2017-09-02 21:54:00  4537.834 -0.06630663              buy   330.727  142521291   8.88e+11           2.04e+13                        923315360
2 2017-09-02 22:29:00  4577.605 -0.05629429              buy   337.804  136524566   8.88e+11           2.03e+13                        918188067
3 2017-09-02 23:04:00  4566.360 -0.05971624              buy   336.938  134845546   8.88e+11           2.01e+13                        910440916
4 2017-09-02 23:39:00  4590.031 -0.05624237              buy   342.929  133910638   8.88e+11           1.99e+13                        901565930
5 2017-09-03 00:14:00  4676.193 -0.03585697             hold   354.171  130678099   8.88e+11           2.01e+13                        922422228
6 2017-09-03 00:49:00  4699.936 -0.03358492             hold   352.299  127557140   8.88e+11           1.99e+13                        910457430
   hash_rate miners_revenue_btc miners_revenue_usd minutes_between_blocks n_blocks_mined n_blocks_total n_btc_mined   n_tx nextretarget
1 7417412092               2395           10839520                   8.00            168         483207    2.10e+11 241558       483839
2 7152504517               2317           10482320                   8.33            162         483208    2.03e+11 236661       483839
3 7240807042               2342           10596900                   8.22            164         483216    2.05e+11 238682       483839
4 7284958305               2352           10642439                   8.14            165         483220    2.06e+11 237159       483839
5 7152504517               2316           10611798                   8.38            162         483223    2.03e+11 237464       483839
6 7064201992               2288           10481960                   8.41            160         483226    2.00e+11 234472       483839
  total_btc_sent total_fees_btc totalbtc trade_volume_btc trade_volume_usd
1       1.62e+14    29597881711 1.65e+15        102451.92        463497285
2       1.60e+14    29202300823 1.65e+15        102451.92        463497285
3       1.60e+14    29234981721 1.65e+15        102451.92        463497285
4       1.58e+14    28991577368 1.65e+15        102451.92        463497285
5       1.58e+14    29179041967 1.65e+15         96216.78        440710136
6       1.57e+14    28844391629 1.65e+15         96216.78        440710136
> str(crypto_data)
'data.frame':   895 obs. of  23 variables:
 $ time                            : POSIXct, format: "2017-09-02 21:54:00" "2017-09-02 22:29:00" "2017-09-02 23:04:00" "2017-09-02 23:39:00" ...
 $ btc_price                       : num  4538 4578 4566 4590 4676 ...
 $ btc_change                      : num  -0.0663 -0.0563 -0.0597 -0.0562 -0.0359 ...
 $ btc_change_label                : Factor w/ 3 levels "buy","hold","sell": 1 1 1 1 2 2 2 2 2 2 ...
 $ eth_price                       : num  331 338 337 343 354 ...
 $ block_size                      : num  1.43e+08 1.37e+08 1.35e+08 1.34e+08 1.31e+08 ...
 $ difficulty                      : num  8.88e+11 8.88e+11 8.88e+11 8.88e+11 8.88e+11 ...
 $ estimated_btc_sent              : num  2.04e+13 2.03e+13 2.01e+13 1.99e+13 2.01e+13 ...
 $ estimated_transaction_volume_usd: num  9.23e+08 9.18e+08 9.10e+08 9.02e+08 9.22e+08 ...
 $ hash_rate                       : num  7.42e+09 7.15e+09 7.24e+09 7.28e+09 7.15e+09 ...
 $ miners_revenue_btc              : num  2395 2317 2342 2352 2316 ...
 $ miners_revenue_usd              : num  10839520 10482320 10596900 10642439 10611798 ...
 $ minutes_between_blocks          : num  8 8.33 8.22 8.14 8.38 8.41 8.26 8.33 8.5 8.69 ...
 $ n_blocks_mined                  : num  168 162 164 165 162 160 157 161 159 156 ...
 $ n_blocks_total                  : num  483207 483208 483216 483220 483223 ...
 $ n_btc_mined                     : num  2.10e+11 2.03e+11 2.05e+11 2.06e+11 2.03e+11 ...
 $ n_tx                            : num  241558 236661 238682 237159 237464 ...
 $ nextretarget                    : num  483839 483839 483839 483839 483839 ...
 $ total_btc_sent                  : num  1.62e+14 1.60e+14 1.60e+14 1.58e+14 1.58e+14 ...
 $ total_fees_btc                  : num  2.96e+10 2.92e+10 2.92e+10 2.90e+10 2.92e+10 ...
 $ totalbtc                        : num  1.65e+15 1.65e+15 1.65e+15 1.65e+15 1.65e+15 ...
 $ trade_volume_btc                : num  102452 102452 102452 102452 96217 ...
 $ trade_volume_usd                : num  4.63e+08 4.63e+08 4.63e+08 4.63e+08 4.41e+08 ...

然后我运行了一个SVM并试图绘制一条ROC曲线：

crypto_linear_svm <- svm(btc_change_label ~ ., data = crypto_trainingDS, method = "C-classification", kernel = "linear")
crypto_linear_svm_pred <- predict(crypto_linear_svm, crypto_testDS[,-3])
linear_crypto_conf_mat <- table(pred = crypto_linear_svm_pred, true = crypto_testDS[,3])
linear_svm_crypto_roc <- plot(multiclass.roc(crypto_testDS$btc_change_label, crypto_linear_svm_pred, direction="<"),
     col="yellow", lwd=3, main="Linear Kernal SVM results, Cryptocurrency Data")

但是，最后一行给出了以下错误：

  

roc.default中的错误（响应，预测器，级别= X，百分比=   百分比，：预测变量必须是数字或有序。

我做错了什么，如何解决这个问题？我有两个不同的数据集，它们具有不同的结构和组织 - 显示的是多类，另一个是二进制（是或否）。我在两者上运行了一个SVM，但是当我尝试绘制ROC时，每个都得到相同的错误。

修改 以下是预测的输出：

> crypto_linear_svm_pred
   3    4    5    6    7    8   14   16   17   19   21   26   29   32   34   36   38   39   45   47   49   53   54   57   59   60   61   63   65 
 buy  buy hold hold hold hold hold hold hold hold hold hold hold hold hold hold hold hold hold hold hold hold hold  buy  buy  buy  buy  buy  buy 
  67   69   71   74   78   86   89   91   92   95   96   97   98  105  111  113  115  116  122  123  124  127  132  135  140  141  156  160  161 
 buy  buy hold hold hold hold hold hold hold hold hold hold sell sell  buy  buy  buy  buy  buy  buy  buy  buy  buy hold hold hold hold  buy hold 
 164  166  170  173  174  175  179  184  188  190  196  208  210  212  214  217  218  219  224  225  227  229  238  240  245  249  259  263  267 
hold hold hold hold hold hold hold hold hold hold hold hold hold hold hold hold hold hold hold hold hold hold hold hold hold hold hold hold  buy 
 273  274  281  282  284  306  307  311  313  315  320  323  324  328  330  332  333  334  336  340  342  343  346  347  349  353  358  361  365 
hold hold  buy  buy  buy  buy  buy  buy  buy  buy  buy  buy  buy  buy  buy  buy  buy  buy  buy  buy  buy  buy  buy  buy  buy  buy  buy  buy  buy 
 374  380  381  382  383  390  392  393  396  399  403  406  407  408  410  435  440  441  444  445  449  453  457  459  460  464  467  468  473 
sell sell sell sell sell sell sell sell sell sell sell sell sell hold hold  buy  buy  buy hold hold hold hold hold hold hold hold hold hold hold 
 483  489  490  492  499  503  511  520  521  530  534  536  538  546  548  553  555  557  558  559  567  571  573  579  581  583  584  586  587 
hold hold hold hold hold hold hold hold hold hold hold hold  buy  buy  buy  buy  buy  buy  buy  buy  buy  buy  buy  buy  buy hold hold hold hold 
 593  595  597  602  603  608  609  614  616  618  628  630  636  639  642  643  645  646  647  648  649  655  660  661  665  668  669  674  675 
hold hold hold hold hold hold hold hold hold hold hold hold hold hold hold hold hold hold hold hold hold hold hold hold hold hold hold hold hold 
 676  680  685  687  688  695  698  703  704  713  715  719  720  722  725  729  737  738  740  744  745  746  752  757  760  762  764  768  771 
hold hold hold hold hold hold sell sell sell sell hold hold hold hold hold hold hold hold hold hold hold hold hold hold hold hold hold hold hold 
 776  778  781  783  784  790  792  805  811  813  814  815  821  822  824  828  829  833  836  837  838  839  843  846  847  848  852  859  861 
hold sell hold hold sell sell sell sell hold sell hold sell hold hold hold hold hold hold hold hold hold hold hold hold hold hold hold hold hold 
 862  865  869  873  879  881  886  895 
hold hold hold hold hold hold hold sell 
Levels: buy hold sell

Answer 1

以下是虹膜数据的示例：

data(iris)
library(e1071)
svm_model = svm(Species~., data = iris)
prob_svm = predict(svm_model, iris)


m.roc = multiclass.roc(iris$Species, as.numeric(prob_svm))

rs <- m.roc[['rocs']]
plot.roc(rs[[1]], lty=4)
sapply(2:length(rs),function(i) lines.roc(rs[[i]],col=i, lty=i))

这种方法计算三条ROC曲线（setosa：versicolor，setosa：virginica one versicolor：virginica）并平均它们的AUC。

它有几个缺陷。将预测类转换为数字是一个。如果可以使用预测的概率，更好的方法是，但是pROC不支持他的行为（我试过）。正如Calimo所指出的那样，ROC是二元分类器的模式，当存在2个以上的类时应该小心使用 我仅以测试数据的预测为例，在评估分类器时不应该这样做，因为它会高估模型的准确性。