Question

我有一个数据集data，包含66个变量和大约200,000个观测值。数据结构的快照如下所示：

$ f37: Factor w/ 10 levels "","1","10","2",..: 2 4 4 2 4 2 4 2 2 4 ...  
$ f38: Factor w/ 3509 levels "","100","1000",..: 1 1 1 1 1 1 1 1 1 1 ...  
$ f39: Factor w/ 15 levels "","1","11","14",..: 7 4 2 4 7 8 15 7 4 4 ...  
$ f40: Factor w/ 15097 levels "","0","1","100",..: 11150 1 5539 1 6710 1650 14680 9235 1 1 ...  
$ f41: num  2156 2156 2156 2156 2156 ...  
$ f42: Factor w/ 14583 levels "","'0503624",..: 11573 1 13170 1 9243 8310 607 1979 1 1 ...  
$ f43: Factor w/ 133655 levels ""," ","(BLANK)",..: 92730 1 2046 1 26944 65286 198 42956 1 1 ...  
$ f44: Factor w/ 145029 levels "","-"," "," 1A8GHB1R76Y123913",..: 49912 1 14207 1 9798 121241 31872 88345 1 1 ...  
$ f45: Factor w/ 124642 levels "","'11291331764964",..: 118266 1 108600 1 14814 3176 46466 99158 1 1 ...  
$ f46: Factor w/ 6524 levels ""," "," HOL 232",..: 85 1 1 1 207 1 1 1  `

数据中有大量的分类变量。我想排除级别大于数字的分类变量， n 。这不仅可以减少计算时间，还可以确保某些机器学习算法（如随机森林）工作。

我尝试过以下代码：

drop_factors <- function(x){
for (j in 1:length(x[1,])) {

        if (is.factor(x[,j])) {
                if(nlevels(x[,j])>32){
                    x[,j]<-NULL
            }
        }

    }
}

然而，当我尝试：

dropfactors(data)

我收到以下错误：

Error in `[.data.frame`(x, , j) : undefined columns selected

我真的很感激任何帮助？

Answer 1

您不需要for循环：

data[sapply(data, nlevels) <= 32]

您收到错误是因为您在for循环期间更改了数据框的列数。因此，j可能会采用大于剩余列数的值。

Answer 2

你可以这样做，以确定超过32个级别的因素：

factor32 <- sapply(data, function(x) class(x) == "factor" & nlevels(x) > 32)
data <- data[, !factor32]

V

使用大量因子删除分类变量

2 个答案: