Question

我正在使用textmineR使LDA模型适合类似于https://cran.r-project.org/web/packages/textmineR/vignettes/c_topic_modeling.html的文档。是否可以获取数据集中每个文档的主题标签？

>library(textmineR)
>data(nih_sample)
> # create a document term matrix 
> dtm <- CreateDtm(doc_vec = nih_sample$ABSTRACT_TEXT,doc_names = 
 nih_sample$APPLICATION_ID, ngram_window = c(1, 2), stopword_vec = 
 c(stopwords::stopwords("en"), stopwords::stopwords(source = "smart")),lower 
 = TRUE, remove_punctuation = TRUE,remove_numbers = TRUE, verbose = FALSE, 
 cpus = 2) 
 >dtm <- dtm[,colSums(dtm) > 2]
 >set.seed(123)
 > model <- FitLdaModel(dtm = dtm, k = 20,iterations = 200,burnin = 
 180,alpha = 0.1, beta = 0.05, optimize_alpha = TRUE, calc_likelihood = 
 TRUE,calc_coherence = TRUE,calc_r2 = TRUE,cpus = 2)

然后将标签添加到模型：

 > model$labels <- LabelTopics(assignments = model$theta > 0.05, dtm = dtm, 
   M = 1)

现在我想要nih_sample$ABSTRACT_TEXT

中每100个文档的主题标签

Answer 1

您是否要通过每个文档中最普遍的主题的标签来对其进行标签？如果是这样，这就是您可以做到的方式：

# convert labels to a data frame so we can merge 
label_df <- data.frame(topic = rownames(model$labels), label = model$labels, stringsAsFactors = FALSE)

# get the top topic for each document
top_topics <- apply(model$theta, 1, function(x) names(x)[which.max(x)][1])

# convert the top topics for each document so we can merge
top_topics <- data.frame(document = names(top_topics), top_topic = top_topics, stringsAsFactors = FALSE)

# merge together. Now each document has a label from its top topic
top_topics <- merge(top_topics, label_df, by.x = "top_topic", by.y = "topic", all.x = TRUE)

这种方法会丢弃您从LDA中获得的一些信息。 LDA的优点之一是每个文档可以包含多个主题。另一个是我们可以看到该文档中每个主题的数量。您可以在这里通过

# set the plot margins to see the labels on the bottom
par(mar = c(8.1,4.1,4.1,2.1))

# barplot the first document's topic distribution with labels
barplot(model$theta[1,], names.arg = model$labels, las = 2)

使用textmineR的LDA模型中每个文档的主题标签

1 个答案: