Apache spark 无法启动部署的spark作业服务器错误org.slf4j.LoggerFactory未找到

我正在尝试在CDH 5.11（spark版本1.6.0）上的群集上使用spark作业服务器 当我尝试在部署的机器上启动spark jobserver时，我会得到以下错误日志

[ERROR] [06/02/2017 15:30:14.966] [JobServer-akka.actor.default-dispatcher-3] [ActorSystem(JobServer)] Uncaught error from thread [JobServer-akka.actor.default-dispatcher-3] shutting down JVM since 'akka.jvm-ex$
java.lang.NoClassDefFoundError: org/slf4j/LoggerFactory
at akka.event.slf4j.Logger$.apply(Slf4jLogger.scala:30)
at akka.event.slf4j.SLF4JLogging$class.log(Slf4jLogger.scala:19)
at akka.event.slf4j.Slf4jLogger.log$lzycompute(Slf4jLogger.scala:54)
at akka.event.slf4j.Slf4jLogger.log(Slf4jLogger.scala:54)
at akka.event.slf4j.Slf4jLogger$$anonfun$receive$1.applyOrElse(Slf4jLogger.scala:80)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)
at akka.actor.ActorCell.invoke(ActorCell.scala:456)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
at akka.dispatch.Mailbox.run(Mailbox.scala:219)
at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
Caused by: java.lang.ClassNotFoundException: org.slf4j.LoggerFactory
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 14 more

Uncaught error from thread [JobServer-akka.actor.default-dispatcher-3] shutting down JVM since 'akka.jvm-exit-on-fatal-error' is enabled for ActorSystem[JobServer]
java.lang.NoClassDefFoundError: org/slf4j/LoggerFactory
at akka.event.slf4j.Logger$.apply(Slf4jLogger.scala:30)
at akka.event.slf4j.SLF4JLogging$class.log(Slf4jLogger.scala:19)
at akka.event.slf4j.Slf4jLogger.log$lzycompute(Slf4jLogger.scala:54)
at akka.event.slf4j.Slf4jLogger.log(Slf4jLogger.scala:54)
at akka.event.slf4j.Slf4jLogger$$anonfun$receive$1.applyOrElse(Slf4jLogger.scala:80)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)
at akka.actor.ActorCell.invoke(ActorCell.scala:456)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
at akka.dispatch.Mailbox.run(Mailbox.scala:219)
at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
Caused by: java.lang.ClassNotFoundException: org.slf4j.LoggerFactory
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 14 more

有没有人遇到过同样的问题或提供过帮助？

这里没有什么

你是如何部署的？ 问题似乎在于SLF4J jar如何从SJS类路径中排除。可能存在版本冲突

并检查UI中的spark相关性。（在那里找到SLF4J！！）

您必须检查scala版本；spark版本和spark作业服务器版本如果冲突

---

在您的情况下，对于spark版本1.6.0；所选scala版本应为2.10.4。（不是像2.11或2.12这样的任何其他版本）

谢谢您的评论，但问题已经解决了！这与您提到的错误相同，但除此之外，CDH中还设置了不正确的类路径，因为我进行了升级，但没有更新spark classpath.txt。更新后一切都很好。现在我还为spark job server使用了不同的repo（它是为几个CDH版本构建的！