Warning: file_get_contents(/data/phpspider/zhask/data//catemap/2/scala/18.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Scala-Spark:从特定节点返回顶点属性_Scala_Graph_Spark Graphx - Fatal编程技术网

Scala-Spark:从特定节点返回顶点属性

Scala-Spark:从特定节点返回顶点属性,scala,graph,spark-graphx,Scala,Graph,Spark Graphx,我有一张图,我想计算最大度数。特别是最大度的顶点,我想知道所有的属性。 以下是代码片段: def max(a: (VertexId, Int), b: (VertexId, Int)): (VertexId, Int) = { if (a._2 > b._2) a else b } val maxDegrees : (VertexId, Int) = graphX.degrees.reduce(max) max: (a: (org.apache.spark.graphx.Ver

我有一张图,我想计算最大度数。特别是最大度的顶点,我想知道所有的属性。 以下是代码片段:

def max(a: (VertexId, Int), b: (VertexId, Int)): (VertexId, Int) = {
    if (a._2 > b._2) a else b
} 

val maxDegrees : (VertexId, Int) = graphX.degrees.reduce(max)
max: (a: (org.apache.spark.graphx.VertexId, Int), b: (org.apache.spark.graphx.VertexId, Int))(org.apache.spark.graphx.VertexId, Int) 
maxDegrees: (org.apache.spark.graphx.VertexId, Int) = (2063726182,56387)

val startVertexRDD = graphX.vertices.filter{case (hash_id, (id, state)) => hash_id == maxDegrees._1}
startVertexRDD.collect()
但它返回了这个异常:

org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 145.0 failed 1 times, most recent failure: Lost task 0.0 in stage 145.0 (TID 5380, localhost, executor driver): scala.MatchError: (1009147972,null) (of class scala.Tuple2)

我想这就是问题所在。在这里:

val startVertexRDD = graphX.vertices.filter{case (hash_id, (id, state)) => hash_id == maxDegrees._1}
所以它试图比较一些像这样的元组

(2063726182,56387)
期待这样的事情:

(hash_id, (id, state))
引发scala.MatchError,因为正在比较(VertextId,Int)的Tuple2与(VertexId,Tuple2(id,state))的Tuple2

对此也要小心:

org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 145.0 failed 1 times, most recent failure: Lost task 0.0 in stage 145.0 (TID 5380, localhost, executor driver): scala.MatchError: (1009147972,null) (of class scala.Tuple2)
具体来说:

scala.MatchError: (1009147972,null)
vertice 1009147972没有计算学位,因此在进行比较时也可能会出现一些问题


希望这有帮助

我认为这就是问题所在。在这里:

val startVertexRDD = graphX.vertices.filter{case (hash_id, (id, state)) => hash_id == maxDegrees._1}
所以它试图比较一些像这样的元组

(2063726182,56387)
期待这样的事情:

(hash_id, (id, state))
引发scala.MatchError,因为正在比较(VertextId,Int)的Tuple2与(VertexId,Tuple2(id,state))的Tuple2

对此也要小心:

org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 145.0 failed 1 times, most recent failure: Lost task 0.0 in stage 145.0 (TID 5380, localhost, executor driver): scala.MatchError: (1009147972,null) (of class scala.Tuple2)
具体来说:

scala.MatchError: (1009147972,null)
vertice 1009147972没有计算学位,因此在进行比较时也可能会出现一些问题


希望这有帮助

我用这段代码检查了是否存在隔离节点:val vertexDegree:VertexRDD[Int]=graphX.degrees val vertexNoDegree=vertexDegree.filter{case(id,degree)=>degree==null}vertexNoDegree.isEmpty()res6:Boolean=true没有隔离节点。。。我不知道该做什么我用这段代码检查了是否有孤立的节点:val vertexDegree:VertexRDD[Int]=graphX.degrees val vertexNoDegree=vertexDegree.filter{case(id,degree)=>degree==null}vertexNoDegree.isEmpty()res6:Boolean=true没有孤立的节点。。。我不知道该怎么办