Apache spark sbt组件0.13.8的spark编译失败

Apache spark sbt组件0.13.8的spark编译失败,apache-spark,sbt,sbt-assembly,Apache Spark,Sbt,Sbt Assembly,在将sbt assembly升级到0.13.8之后,我正在尝试构建一个spark应用程序。这是我现在拥有的build.sbt的内容。但这不起作用。这似乎是默认合并策略的问题。我发现大约有600多个重复数据消除错误 name := "volumeApp" version := "0.0.1" scalaVersion := "2.10.4" libraryDependencies ++= Seq( "org.apache.spark" %% "spark-streaming-kaf

在将sbt assembly升级到0.13.8之后,我正在尝试构建一个spark应用程序。这是我现在拥有的build.sbt的内容。但这不起作用。这似乎是默认合并策略的问题。我发现大约有600多个重复数据消除错误

name := "volumeApp"

version := "0.0.1"

scalaVersion := "2.10.4" 

libraryDependencies ++= Seq(
    "org.apache.spark" %% "spark-streaming-kafka" % "1.3.1",
    "org.apache.spark" %% "spark-core" % "1.3.1",
    "org.apache.spark" %% "spark-streaming" % "1.3.1",
    "org.apache.kafka" %% "kafka" % "0.8.2.1",
    "com.datastax.spark" %% "spark-cassandra-connector" % "1.3.0-M1",
    "com.github.vonnagy" %% "service-container-metrics-reporting" % "1.0.1" exclude("com.codahale.metrics", "metrics-core"),
    "joda-time" % "joda-time" % "2.7",
    "log4j" % "log4j" % "1.2.14"
)

assemblyJarName in assembly := "inventoryVolume.jar"

assemblyMergeStrategy in assembly := {
    case PathList("META-INF", "jboss-beans.xml") => MergeStrategy.first
    case PathList("META-INF", "mailcap") => MergeStrategy.discard
    case PathList("META-INF", "maven", "org.slf4j", "slf4j-api", xa @ _*) => MergeStrategy.rename
    case PathList("META-INF", "ECLIPSEF.RSA") => MergeStrategy.discard
    case PathList("META-INF", "mimetypes.default") => MergeStrategy.first
    case PathList("com", "datastax", "driver", "core", "Driver.properties") => MergeStrategy.last
    case PathList("com", "esotericsoftware", "minlog", xx @ _*) => MergeStrategy.first
    case PathList("plugin.properties") => MergeStrategy.discard
    case PathList("javax", "activation", xw @ _*) => MergeStrategy.first
    case PathList("org", "apache", "hadoop", "yarn", xv @ _*) => MergeStrategy.first
    case PathList("org", "apache", "commons", xz @ _*) => MergeStrategy.first
    case PathList("org", "jboss", "netty", ya @ _*) => MergeStrategy.first
    case x =>
        val baseStrategy = (assemblyMergeStrategy in assembly).value
        baseStrategy(x)
}
我把错误附在这里了
有人能帮帮我吗。。我对构建工具是完全陌生的。我试着阅读sbt指南,但很快就迷路了。

似乎您正在尝试构建一个包含多个版本的akka actor系统的程序集jar。有几种可能的解决方案,一种是将其从依赖项中排除(有关想法,请参阅),或者改变合并策略,但我建议使用排除依赖项的方法。

感谢您指出@Holden在排除对akka_actor的依赖项时,我无法理解其语法。。我的build.sbt中的一行已经有一个exclude语句没有任何效果,因为我仍然收到重复数据消除错误w.r.t metrics core。对我来说,使用MergeStrategy.first的MergeStrategy是一个默认情况。我稍后会尝试找出一个更好的解决方案。但奇怪的是,当我使用sbt assembly 0.11时,并没有任何问题+