Warning: file_get_contents(/data/phpspider/zhask/data//catemap/3/apache-spark/6.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Apache spark 如何使用Spark LinearSVC型号获得最佳功能?_Apache Spark_Machine Learning_Feature Selection - Fatal编程技术网

Apache spark 如何使用Spark LinearSVC型号获得最佳功能?

Apache spark 如何使用Spark LinearSVC型号获得最佳功能?,apache-spark,machine-learning,feature-selection,Apache Spark,Machine Learning,Feature Selection,我试图使用ChiSqSelector确定Spark 2.2 LSVCModel的最佳功能,因此: import org.apache.spark.ml.feature.ChiSqSelector val chiSelector = new ChiSqSelector().setNumTopFeatures(5). setFeaturesCol("features"). setLabelCol("label").setOutputCol("selectedFeatures") val

我试图使用ChiSqSelector确定Spark 2.2 LSVCModel的最佳功能,因此:

import org.apache.spark.ml.feature.ChiSqSelector
val chiSelector = new ChiSqSelector().setNumTopFeatures(5).
   setFeaturesCol("features").
   setLabelCol("label").setOutputCol("selectedFeatures")

val pipeline = new Pipeline().setStages(Array(labelIndexer, monthIndexer, hashingTF
   , idf, va, featureIndexer,  chiSelector, lsvc, labelConverter))

val model = pipeline.fit(training)
val importantFeatures = model.selectedFeatures

import org.apache.spark.ml.classification.LinearSVCModel
val LSVCModel= model.stages(6).asInstanceOf[org.apache.spark.ml.classification.
   LinearSVCModel]

val importantFeatures = LSVCModel.selectedFeatures
这就产生了错误:

<console>:180: error: value selectedFeatures is not a member of 
org.apache.spark.ml.classification.LinearSVCModel
   val importantFeatures = LSVCModel.selectedFeatures
:180:错误:value selectedFeatures不是
org.apache.spark.ml.classification.LinearSVCModel
val importantFeatures=LSVCModel.selectedFeatures

这种型号可以使用ChiSqSelector吗?如果没有,是否有其他选择?

线性SVC不会进行任何功能选择。您应该从管道中提取
ChiSqSelectorModel
,而不是
LinearSVCModel

import org.apache.spark.ml.feature.ChiSqSelectorModel
val chiSqModel = model.stages(6).asInstanceOf[ChiSqSelectorModel]

val importantFeatures = chiSqModel.selectedFeatures

你用错型号了。它不是
LinearSVCModel
。谢谢,我会试一试。效果很好,但如何获得实际功能
selectedFeatures
只是给了我一些索引。@schoon:它会给你
features
列中的功能的索引。因此,要知道选择了哪些特征,您应该查看如何在本专栏中构建向量。