Sbt 如何在Play2.2.3项目中使用Spark 1.2.0,因为它在NoSuchMethodError:akka.util.Helpers中失败?

Sbt 如何在Play2.2.3项目中使用Spark 1.2.0,因为它在NoSuchMethodError:akka.util.Helpers中失败?,sbt,apache-spark,playframework-2.2,typesafe-activator,Sbt,Apache Spark,Playframework 2.2,Typesafe Activator,您是否曾经在Play framework方面遇到过问题?在我的例子中,首先我已经在一个jar中构建了所有组件:spark-assebmly-1.2.0-hadoop2.4.0.jar,spark可以在一个shell中完美地工作。但有两个问题: 我应该在Play_项目中使用这个组装的Spark_罐子吗?如何使用??因为我试图将它转移到lib_directiry,但这无助于提供任何Spark_导入 如果我定义Spark库,比如:“org.apache.Spark”%%“Spark核心”%%“1.2.

您是否曾经在Play framework方面遇到过问题?在我的例子中,首先我已经在一个jar中构建了所有组件:
spark-assebmly-1.2.0-hadoop2.4.0.jar
,spark可以在一个shell中完美地工作。但有两个问题:

  • 我应该在Play_项目中使用这个组装的Spark_罐子吗?如何使用??因为我试图将它转移到lib_directiry,但这无助于提供任何Spark_导入

  • 如果我定义Spark库,比如:
    “org.apache.Spark”%%“Spark核心”%%“1.2.0”

  • 播放框架代码

    Build.scala

    val-appDependencies=Seq(
    jdbc
    ,“org.apache.spark”%%“spark流媒体”%%“1.2.0”
    ,“org.apache.spark”%%“spark核心”%%“1.2.0”
    ,“org.apache.spark”%%“spark sql”%%“1.2.0”
    
    TestEntity.scala

    包模型
    导入org.apache.spark.SparkContext
    导入org.apache.spark.SparkContext_
    导入org.apache.hadoop.conf.Configuration
    导入模型.SparkMain
    导入org.apache.spark.rdd.rdd
    导入org.apache.spark.SparkContext_
    对象测试性{
    val TestEntityPath=“/home/t/PROD/dict/TestEntity.txt”
    val TestEntitySpark=SparkMain.sc.textFile(TestEntityPath,4).cache
    val TestEntityData=TestEntitySpark.flatMap(u.split(“,”))
    def getFive():Seq[字符串]={
    println(“TestEntity.getFive”)
    TestEntityData.take(5)
    }
    }
    
    SparkMain.scala

    包模型
    导入org.apache.spark.SparkContext
    导入org.apache.spark.SparkContext_
    导入org.apache.hadoop.conf.Configuration
    导入org.apache.spark.rdd.rdd
    导入org.apache.spark.SparkContext_
    导入org.apache.spark.streaming.{Seconds,StreamingContext}
    导入StreamingContext_
    导入org.apache.spark.sql.SQLContext
    导入org.apache.spark.SparkConf
    物体火花{
    val driverPort=8080
    val driverHost=“localhost”
    val conf=new SparkConf(false)//跳过加载外部设置
    .setMaster(“本地[4]”//使用足够的线程在本地运行
    .setAppName(“firstSparkApp”)
    .set(“spark.logConf”、“true”)
    .set(“spark.driver.port”,s“$driverPort”)
    .set(“spark.driver.host”,s“$driverHost”)
    .set(“spark.akka.logLifecycleEvents”,“true”)
    val sc=新的SparkContext(配置)
    }
    
    和控制器代码,使用Spark的东西:

    def测试=操作{
    隐式请求=>{
    val chk=TestEntity.getFive
    好的(“它起作用”)
    }
    }
    
    ..在运行时a中出现以下错误:

    [info] o.a.s.SparkContext - Spark configuration:
    spark.akka.logLifecycleEvents=true
    spark.app.name=firstSparkApp
    spark.driver.host=localhost
    spark.driver.port=8080
    spark.logConf=true
    spark.master=local[4]
    [warn] o.a.s.u.Utils - Your hostname, uisprk resolves to a loopback address: 127.0.1.1; using 10.0.2.15 instead (on interface eth0)
    [warn] o.a.s.u.Utils - Set SPARK_LOCAL_IP if you need to bind to another address
    [info] o.a.s.SecurityManager - Changing view acls to: t
    [info] o.a.s.SecurityManager - Changing modify acls to: t
    [info] o.a.s.SecurityManager - SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(t); users with modify permissions: Set(t)
    [error] application -
    
    ! @6l039e8d5 - Internal server error, for (GET) [/ui] ->
    
    play.api.Application$$anon$1: Execution exception[[RuntimeException: java.lang.NoSuchMethodError: akka.util.Helpers$.ConfigOps(Lcom/typesafe/config/Config;)Lcom/typesafe/config/Config;]]
            at play.api.Application$class.handleError(Application.scala:293) ~[play_2.10-2.2.3.jar:2.2.3]
            at play.api.DefaultApplication.handleError(Application.scala:399) [play_2.10-2.2.3.jar:2.2.3]
            at play.core.server.netty.PlayDefaultUpstreamHandler$$anonfun$13$$anonfun$apply$1.applyOrElse(PlayDefaultUpstreamHandler.scala:166) [play_2.10-2.2.3.jar:2.2.3]
            at play.core.server.netty.PlayDefaultUpstreamHandler$$anonfun$13$$anonfun$apply$1.applyOrElse(PlayDefaultUpstreamHandler.scala:163) [play_2.10-2.2.3.jar:2.2.3]
            at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:33) [scala-library-2.10.4.jar:na]
            at scala.util.Failure$$anonfun$recover$1.apply(Try.scala:185) [scala-library-2.10.4.jar:na]
    Caused by: java.lang.RuntimeException: java.lang.NoSuchMethodError: akka.util.Helpers$.ConfigOps(Lcom/typesafe/config/Config;)Lcom/typesafe/config/Config;
            at play.api.mvc.ActionBuilder$$anon$1.apply(Action.scala:314) ~[play_2.10-2.2.3.jar:2.2.3]
            at play.api.mvc.Action$$anonfun$apply$1$$anonfun$apply$4$$anonfun$apply$5.apply(Action.scala:109) ~[play_2.10-2.2.3.jar:2.2.3]
            at play.api.mvc.Action$$anonfun$apply$1$$anonfun$apply$4$$anonfun$apply$5.apply(Action.scala:109) ~[play_2.10-2.2.3.jar:2.2.3]
            at play.utils.Threads$.withContextClassLoader(Threads.scala:18) ~[play_2.10-2.2.3.jar:2.2.3]
            at play.api.mvc.Action$$anonfun$apply$1$$anonfun$apply$4.apply(Action.scala:108) ~[play_2.10-2.2.3.jar:2.2.3]
            at play.api.mvc.Action$$anonfun$apply$1$$anonfun$apply$4.apply(Action.scala:107) ~[play_2.10-2.2.3.jar:2.2.3]
    Caused by: java.lang.NoSuchMethodError: akka.util.Helpers$.ConfigOps(Lcom/typesafe/config/Config;)Lcom/typesafe/config/Config;
            at akka.remote.RemoteSettings.<init>(RemoteSettings.scala:48) ~[akka-remote_2.10-2.3.4-spark.jar:na]
            at akka.remote.RemoteActorRefProvider.<init>(RemoteActorRefProvider.scala:114) ~[akka-remote_2.10-2.3.4-spark.jar:na]
            at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[na:1.7.0_72]
            at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) ~[na:1.7.0_72]
            at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[na:1.7.0_72]
            at java.lang.reflect.Constructor.newInstance(Constructor.java:526) ~[na:1.7.0_72]
    
    [info]o.a.s.SparkContext-Spark配置:
    spark.akka.logLifecycleEvents=true
    spark.app.name=firstSparkApp
    spark.driver.host=localhost
    火花驱动器端口=8080
    spark.logConf=true
    spark.master=本地[4]
    [警告]o.a.s.u.Utils-您的主机名,uisprk解析为环回地址:127.0.1.1;改为使用10.0.2.15(在接口eth0上)
    [警告]o.a.s.u.Utils-如果需要绑定到其他地址,请设置SPARK_LOCAL_IP
    [信息]o.a.s.SecurityManager-将视图ACL更改为:t
    [信息]o.a.s.SecurityManager-将修改ACL更改为:t
    [信息]o.a.s.SecurityManager-SecurityManager:禁用身份验证;禁用ui ACL;具有查看权限的用户:设置(t);具有修改权限的用户:设置(t)
    [错误]应用程序-
    !@6l039e8d5-内部服务器错误,用于(GET)[/ui]->
    play.api.Application$$anon$1:执行异常[[RuntimeException:java.lang.NoSuchMethodError:akka.util.Helpers$.ConfigOps(Lcom/typesafe/config/config;)Lcom/typesafe/config/config;]]
    在play.api.Application$class.handleError(Application.scala:293)~[play_2.10-2.2.3.jar:2.2.3]
    play.api.DefaultApplication.handleError(Application.scala:399)[play_2.10-2.2.3.jar:2.2.3]
    在play.core.server.netty.playdaultUpstreamHandler$$anonfun$13$$anonfun$apply$1.applyOrElse(playdaultUpstreamHandler.scala:166)[play_2.10-2.2.3.jar:2.2.3]
    在play.core.server.netty.playdaultUpstreamHandler$$anonfun$13$$anonfun$apply$1.applyOrElse(playdaultUpstreamHandler.scala:163)[play_2.10-2.2.3.jar:2.2.3]
    在scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:33)[scala-library-2.10.4.jar:na]
    在scala.util.Failure$$anonfun$recover$1.apply(Try.scala:185)[scala-library-2.10.4.jar:na]
    原因:java.lang.RuntimeException:java.lang.NoSuchMethodError:akka.util.Helpers$.ConfigOps(Lcom/typesafe/config/config;)Lcom/typesafe/config/config;
    在play.api.mvc.ActionBuilder$$anon$1.apply(Action.scala:314)~[play_2.10-2.2.3.jar:2.2.3]
    在play.api.mvc.Action$$anonfun$apply$1$$anonfun$apply$4$$anonfun$apply$5.apply(Action.scala:109)~[play_2.10-2.2.3.jar:2.2.3]
    在play.api.mvc.Action$$anonfun$apply$1$$anonfun$apply$4$$anonfun$apply$5.apply(Action.scala:109)~[play_2.10-2.2.3.jar:2.2.3]
    在play.utils.Threads$.withContextClassLoader(Threads.scala:18)~[play_2.10-2.2.3.jar:2.2.3]
    在play.api.mvc.Action$$anonfun$apply$1$$anonfun$apply$4.apply(Action.scala:108)~[play_2.10-2.2.3.jar:2.2.3]
    在play.api.mvc.Action$$anonfun$apply$1$$anonfun$apply$4.apply(Action.scala:107)~[play_2.10-2.2.3.jar:2.2.3]
    原因:java.lang.NoSuchMethodError:akka.util.Helpers$.ConfigOps(Lcom/typesafe/config/config;)Lcom/typesafe/config;
    在akka.remote.RemoteSettings.(RemoteSettings.scala:48)~[akka-remote_2.10-2.3.4-spark.jar:na]
    在akka.remote.RemoteActorRefProvider.(RemoteActorRefProvider.scala:114)~[akka-remote_2.10-2.3.4-spark.jar:na]
    在sun.reflect.NativeConstructorAccessorImpl.newInstance0(本机方法)~[na:1.7.0\u 72]
    在sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)~[na:1.7.0\U 72]
    在sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)~[na:1.7.072]
    在java.lang.reflect.Constructor.newInstance(C