Apache spark 如何使用Spark LinearSVC型号获得最佳功能?
我试图使用ChiSqSelector确定Spark 2.2 LSVCModel的最佳功能,因此:Apache spark 如何使用Spark LinearSVC型号获得最佳功能?,apache-spark,machine-learning,feature-selection,Apache Spark,Machine Learning,Feature Selection,我试图使用ChiSqSelector确定Spark 2.2 LSVCModel的最佳功能,因此: import org.apache.spark.ml.feature.ChiSqSelector val chiSelector = new ChiSqSelector().setNumTopFeatures(5). setFeaturesCol("features"). setLabelCol("label").setOutputCol("selectedFeatures") val
import org.apache.spark.ml.feature.ChiSqSelector
val chiSelector = new ChiSqSelector().setNumTopFeatures(5).
setFeaturesCol("features").
setLabelCol("label").setOutputCol("selectedFeatures")
val pipeline = new Pipeline().setStages(Array(labelIndexer, monthIndexer, hashingTF
, idf, va, featureIndexer, chiSelector, lsvc, labelConverter))
val model = pipeline.fit(training)
val importantFeatures = model.selectedFeatures
import org.apache.spark.ml.classification.LinearSVCModel
val LSVCModel= model.stages(6).asInstanceOf[org.apache.spark.ml.classification.
LinearSVCModel]
val importantFeatures = LSVCModel.selectedFeatures
这就产生了错误:
<console>:180: error: value selectedFeatures is not a member of
org.apache.spark.ml.classification.LinearSVCModel
val importantFeatures = LSVCModel.selectedFeatures
:180:错误:value selectedFeatures不是
org.apache.spark.ml.classification.LinearSVCModel
val importantFeatures=LSVCModel.selectedFeatures
这种型号可以使用ChiSqSelector吗?如果没有,是否有其他选择?线性SVC不会进行任何功能选择。您应该从管道中提取
ChiSqSelectorModel
,而不是LinearSVCModel
import org.apache.spark.ml.feature.ChiSqSelectorModel
val chiSqModel = model.stages(6).asInstanceOf[ChiSqSelectorModel]
val importantFeatures = chiSqModel.selectedFeatures
你用错型号了。它不是
LinearSVCModel
。谢谢,我会试一试。效果很好,但如何获得实际功能selectedFeatures
只是给了我一些索引。@schoon:它会给你features
列中的功能的索引。因此,要知道选择了哪些特征,您应该查看如何在本专栏中构建向量。