在R(插入符号)中重新运行preProcess()、predict()和train()时,模型精度不同

在R(插入符号)中重新运行preProcess()、predict()和train()时,模型精度不同,r,machine-learning,r-caret,glm,R,Machine Learning,R Caret,Glm,下面的数据只是一个示例,我对这些数据或任何数据的操作感到困惑: library(caret) set.seed(3433) data(AlzheimerDisease) complete <- data.frame(diagnosis, predictors) in_train <- createDataPartition(complete$diagnosis, p = 0.75)[[1]] training <- complete[in_train,] testing <

下面的数据只是一个示例,我对这些数据或任何数据的操作感到困惑:

library(caret)
set.seed(3433)
data(AlzheimerDisease)
complete <- data.frame(diagnosis, predictors)
in_train <- createDataPartition(complete$diagnosis, p = 0.75)[[1]]
training <- complete[in_train,]
testing <- complete[-in_train,]
predIL <- grep("^IL", names(training))
smalltrain <- training[, c(1, predIL)]

fit_noPCA <- train(diagnosis ~ ., method = "glm", data = smalltrain)
pre_proc_obj <- preProcess(smalltrain[,-1], method = "pca", thresh = 0.8)
smalltrainsPCs <- predict(pre_proc_obj, smalltrain[,-1])
fit_PCA <- train(x = smalltrainsPCs, y = smalltrain$diagnosis, method = "glm")
fit_noPCA$results$Accuracy
fit_PCA$results$Accuracy

然后每次我重新运行这6行,我得到不同的精度值。为什么会这样?是因为我没有重置种子吗?即使如此,这一过程的内在随机性在哪里

默认情况下,模型使用引导进行训练,您可以在此处看到:

library(caret)
library(AppliedPredictiveModeling)

> fit_noPCA
Generalized Linear Model 

251 samples
 12 predictor
  2 classes: 'Impaired', 'Control' 

No pre-processing
Resampling: Bootstrapped (25 reps) 
Summary of sample sizes: 251, 251, 251, 251, 251, 251, ... 
Resampling results:

  Accuracy   Kappa     
  0.6870006  0.04107016
因此,对于每一个
序列
,引导的样本将不同,为了得到相同的结果,您可以在运行序列之前设置种子:

set.seed(111)
fit_PCA <- train(x = smalltrainsPCs, y = smalltrain$diagnosis, method = "glm",trControl=trainControl(method="boot",number=100))
fit_PCA$results$Accuracy
[1] 0.6983512

set.seed(112)
fit_PCA <- train(x = smalltrainsPCs, y = smalltrain$diagnosis, method = "glm",trControl=trainControl(method="boot",number=100))
fit_PCA$results$Accuracy
[1] 0.6991537

set.seed(111)
fit_PCA <- train(x = smalltrainsPCs, y = smalltrain$diagnosis, method = "glm",trControl=trainControl(method="boot",number=100))
fit_PCA$results$Accuracy
[1] 0.6983512
set.seed(111)

这些值是什么?它们是变化很大还是只在最后几位小数中变化?它们分别是0.6688098和0.7012564、0.6691034和0.7106764或0.6932403和0.687635,用于
fit_noPCA
fit_PCA
。所以我认为它们之间的差异很大,而且并不总是一种适合比另一种好。谢谢,这很有道理
set.seed(111)
fit_PCA <- train(x = smalltrainsPCs, y = smalltrain$diagnosis, method = "glm",trControl=trainControl(method="boot",number=100))
fit_PCA$results$Accuracy
[1] 0.6983512

set.seed(112)
fit_PCA <- train(x = smalltrainsPCs, y = smalltrain$diagnosis, method = "glm",trControl=trainControl(method="boot",number=100))
fit_PCA$results$Accuracy
[1] 0.6991537

set.seed(111)
fit_PCA <- train(x = smalltrainsPCs, y = smalltrain$diagnosis, method = "glm",trControl=trainControl(method="boot",number=100))
fit_PCA$results$Accuracy
[1] 0.6983512