Scala 如何通过apache spark graphX获取SSSP实际路径?
我在spark站点上运行了单源最短路径(SSSP)示例,如下所示: 代码(scala):Scala 如何通过apache spark graphX获取SSSP实际路径?,scala,apache-spark,spark-graphx,Scala,Apache Spark,Spark Graphx,我在spark站点上运行了单源最短路径(SSSP)示例,如下所示: 代码(scala): object Pregel\u SSSP{ def main(参数:数组[字符串]){ val sc=新的SparkContext(“本地”、“Allen Pregel测试”、System.getenv(“SPARK_HOME”)、SparkContext.jarOfClass(this.getClass)) //具有包含距离的边属性的图 val图形:图形[Int,Double]= GraphGenera
object Pregel\u SSSP{
def main(参数:数组[字符串]){
val sc=新的SparkContext(“本地”、“Allen Pregel测试”、System.getenv(“SPARK_HOME”)、SparkContext.jarOfClass(this.getClass))
//具有包含距离的边属性的图
val图形:图形[Int,Double]=
GraphGenerators.logNormalGraph(sc,numVertices=5).MapEdge(e=>e.attr.toDouble)
图.边.foreach(println)
val sourceId:VertexId=0//最终源
//初始化图形,使除根之外的所有顶点都具有无穷远的距离。
val initialGraph=graph.mapVertices((id,)=>if(id==sourceId)0.0 else Double.PositiveInfinity)
val sssp=initialGraph.pregel(Double.PositiveInfinity、Int.MaxValue、EdgeDirection.Out)(
//顶点程序
(id,dist,newDist)=>math.min(dist,newDist),
//发送消息
三重态=>{
if(triplet.srcatr+triplet.attr数学最小值(a,b))
println(sssp.vertices.collect.mkString(“\n”))
}
}
sourceId:0获取结果:
(0,0.0)
(4,2.0)
(2,1.0)
(3,1.0)
(1,2.0)
但我需要如下所示的实际路径:
=>
0->0,0
0->2,1
0->3,1
0->2->4,2
0->3->1,2
如何通过spark graphX获得SSSP实际路径
有人给我一些提示吗
谢谢你的帮助
为了存储最短路径长度和实际路径,您必须修改算法。 所以,不应该将
Double
存储为顶点的属性,而应该存储tuple:(Double,List[VertexId])
也许这段代码对你有用
object Pregel_SSSP {
def main(args: Array[String]) {
val sc = new SparkContext("local", "Allen Pregel Test", System.getenv("SPARK_HOME"), SparkContext.jarOfClass(this.getClass))
// A graph with edge attributes containing distances
val graph: Graph[Int, Double] =
GraphGenerators.logNormalGraph(sc, numVertices = 5).mapEdges(e => e.attr.toDouble)
graph.edges.foreach(println)
val sourceId: VertexId = 0 // The ultimate source
// Initialize the graph such that all vertices except the root have distance infinity.
val initialGraph : Graph[(Double, List[VertexId]), Double] = graph.mapVertices((id, _) => if (id == sourceId) (0.0, List[VertexId](sourceId)) else (Double.PositiveInfinity, List[VertexId]()))
val sssp = initialGraph.pregel((Double.PositiveInfinity, List[VertexId]()), Int.MaxValue, EdgeDirection.Out)(
// Vertex Program
(id, dist, newDist) => if (dist._1 < newDist._1) dist else newDist,
// Send Message
triplet => {
if (triplet.srcAttr._1 < triplet.dstAttr._1 - triplet.attr ) {
Iterator((triplet.dstId, (triplet.srcAttr._1 + triplet.attr , triplet.srcAttr._2 :+ triplet.dstId)))
} else {
Iterator.empty
}
},
//Merge Message
(a, b) => if (a._1 < b._1) a else b)
println(sssp.vertices.collect.mkString("\n"))
}
}
object Pregel\u SSSP{
def main(参数:数组[字符串]){
val sc=新的SparkContext(“本地”、“Allen Pregel测试”、System.getenv(“SPARK_HOME”)、SparkContext.jarOfClass(this.getClass))
//具有包含距离的边属性的图
val图形:图形[Int,Double]=
GraphGenerators.logNormalGraph(sc,numVertices=5).MapEdge(e=>e.attr.toDouble)
图.边.foreach(println)
val sourceId:VertexId=0//最终源
//初始化图形,使除根之外的所有顶点都具有无穷远的距离。
val initialGraph:Graph[(Double,List[VertexId]),Double]=Graph.mapVertexts((id,)=>if(id==sourceId)(0.0,List[VertexId](sourceId))else(Double.PositiveInfinity,List[VertexId]())
val sssp=initialGraph.pregel((Double.PositiveInfinity,List[VertexId]()),Int.MaxValue,EdgeDirection.Out)(
//顶点程序
(id,dist,newDist)=>if(dist.\u 1{
if(triplet.srcAttr._1如果(a._1
可能这是一个过时的答案,但看看这个解决方案在上面的代码中,我尝试的是从源节点到所有节点的最短路径。但是,是否可以修改上述代码以获得单个源节点和单个目标节点之间的最短路径?我不想过滤输出。
object Pregel_SSSP {
def main(args: Array[String]) {
val sc = new SparkContext("local", "Allen Pregel Test", System.getenv("SPARK_HOME"), SparkContext.jarOfClass(this.getClass))
// A graph with edge attributes containing distances
val graph: Graph[Int, Double] =
GraphGenerators.logNormalGraph(sc, numVertices = 5).mapEdges(e => e.attr.toDouble)
graph.edges.foreach(println)
val sourceId: VertexId = 0 // The ultimate source
// Initialize the graph such that all vertices except the root have distance infinity.
val initialGraph : Graph[(Double, List[VertexId]), Double] = graph.mapVertices((id, _) => if (id == sourceId) (0.0, List[VertexId](sourceId)) else (Double.PositiveInfinity, List[VertexId]()))
val sssp = initialGraph.pregel((Double.PositiveInfinity, List[VertexId]()), Int.MaxValue, EdgeDirection.Out)(
// Vertex Program
(id, dist, newDist) => if (dist._1 < newDist._1) dist else newDist,
// Send Message
triplet => {
if (triplet.srcAttr._1 < triplet.dstAttr._1 - triplet.attr ) {
Iterator((triplet.dstId, (triplet.srcAttr._1 + triplet.attr , triplet.srcAttr._2 :+ triplet.dstId)))
} else {
Iterator.empty
}
},
//Merge Message
(a, b) => if (a._1 < b._1) a else b)
println(sssp.vertices.collect.mkString("\n"))
}
}