Java 无法启动Apache Spark 2.1.1的工作节点_Java_Apache Spark

Java 无法启动Apache Spark 2.1.1的工作节点

java apache-spark

Java 无法启动Apache Spark 2.1.1的工作节点,java,apache-spark,Java,Apache Spark,我无法启动Apache Spark 2.1.1的工作节点（我有单节点设置，我的主节点和工作节点将在同一台主机上运行），以下是日志： $ ./start-all.sh 启动org.apache.spark.deploy.master.master，登录到 /home/kedar/software/spark/logs/spark-kedar-org.apache.spark.deploy.master.master-1-myhost.out localhost:启动org.apache.spa

我无法启动Apache Spark 2.1.1的工作节点（我有单节点设置，我的主节点和工作节点将在同一台主机上运行），以下是日志：

$ ./start-all.sh

启动org.apache.spark.deploy.master.master，登录到 /home/kedar/software/spark/logs/spark-kedar-org.apache.spark.deploy.master.master-1-myhost.out localhost:启动org.apache.spark.deploy.worker.worker，登录到 /home/kedar/software/spark/logs/spark-kedar-org.apache.spark.deploy.worker.worker-1-myhost.out localhost:未能启动：nice-n 0 /home/kedar/software/spark/bin/spark类 org.apache.spark.deploy.worker.worker——webui端口8081 spark://myhost:7077 localhost:at loadClass（ClassLoader.java:358）localhost:at launcher.LaunchHelper.checkAndLoadMain（LaunchHelper.java:482） localhost:完全登录 /home/kedar/software/spark/logs/spark-kedar-org.apache.spark.deploy.worker.worker-1-myhost.out

以下是日志：

$ cat /home/kedar/software/spark/logs/spark-kedar-org.apache.spark.deploy.worker.Worker-1-myhost.out

线程“main”java.lang.UnsupportedClassVersionError中出现异常： org/apache/spark/launcher/Main：不支持的major.minor版本52.0 位于java.lang.ClassLoader.defineClass1（本机方法） java.lang.ClassLoader.defineClass（ClassLoader.java:803）位于 security.SecureClassLoader.defineClass（SecureClassLoader.java:142）位于java.net.URLClassLoader.defineClass（URLClassLoader.java:442）访问$100（URLClassLoader.java:64） java.net.URLClassLoader$1.run（URLClassLoader.java:354）位于 java.net.URLClassLoader$1.run（URLClassLoader.java:348）位于 java.security.AccessController.doPrivileged（本机方法）位于 java.net.URLClassLoader.findClass（URLClassLoader.java:347）位于 loadClass（ClassLoader.java:425）位于 sun.misc.Launcher$AppClassLoader.loadClass（Launcher.java:312）位于 loadClass（ClassLoader.java:358）位于 launcher.LaunchHelper.checkAndLoadMain（LaunchHelper.java:482）

感谢您的帮助！谢谢

问题是我的java替代方案指向了错误的java版本，下面是在ubuntu上解决问题的步骤，正如我们所看到的，Spark也成功启动了：

kedar@myhost:/opt$ sudo update-alternatives --install "/usr/bin/javac" "javac" "/opt/jdk1.8.0_72/bin/javac"
javac
kedar@myhost:/opt$ sudo update-alternatives --install "/usr/bin/javac" "javac" "/opt/jdk1.8.0_72/bin/javac" 1
kedar@myhost:/opt$ ll javac
ls: cannot access javac: No such file or directory
kedar@myhost:/opt$ sudo update-alternatives --config java
There is only one alternative in link group java (providing /usr/bin/java): /usr/lib/jvm/java-7-openjdk-amd64/jre/bin/java
Nothing to configure.
kedar@myhost:/opt$ sudo update-alternatives --config javac
There are 2 choices for the alternative javac (providing /usr/bin/javac).

  Selection    Path                                         Priority   Status
------------------------------------------------------------
* 0            /usr/lib/jvm/java-7-openjdk-amd64/bin/javac   1071      auto mode
  1            /opt/jdk1.8.0_72/bin/javac                    1         manual mode
  2            /usr/lib/jvm/java-7-openjdk-amd64/bin/javac   1071      manual mode

Press enter to keep the current choice[*], or type selection number: 1       
update-alternatives: using /opt/jdk1.8.0_72/bin/javac to provide /usr/bin/javac (javac) in manual mode
kedar@myhost:/opt$ sudo update-alternatives --config javac
There are 2 choices for the alternative javac (providing /usr/bin/javac).

  Selection    Path                                         Priority   Status
------------------------------------------------------------
  0            /usr/lib/jvm/java-7-openjdk-amd64/bin/javac   1071      auto mode
* 1            /opt/jdk1.8.0_72/bin/javac                    1         manual mode
  2            /usr/lib/jvm/java-7-openjdk-amd64/bin/javac   1071      manual mode

Press enter to keep the current choice[*], or type selection number: kedar@myhost:/opt$ 
kedar@myhost:/opt$ 
kedar@myhost:/opt$ 
kedar@myhost:/opt$ sudo update-alternatives --install "/usr/bin/java" "java" "/opt/jdk1.8.0_72/bin/java" 1
kedar@myhost:/opt$ ll javac
ls: cannot access javac: No such file or directory
kedar@myhost:/opt$ sudo update-alternatives --config java
There are 2 choices for the alternative java (providing /usr/bin/java).

  Selection    Path                                            Priority   Status
------------------------------------------------------------
* 0            /usr/lib/jvm/java-7-openjdk-amd64/jre/bin/java   1071      auto mode
  1            /opt/jdk1.8.0_72/bin/java                        1         manual mode
  2            /usr/lib/jvm/java-7-openjdk-amd64/jre/bin/java   1071      manual mode

Press enter to keep the current choice[*], or type selection number: 1
update-alternatives: using /opt/jdk1.8.0_72/bin/java to provide /usr/bin/java (java) in manual mode
kedar@myhost:/opt$ cd
kedar@myhost:~$ cd $SPARK_HOME
kedar@myhost:~/software/spark$ ./sbin/stop-all.sh 
localhost: no org.apache.spark.deploy.worker.Worker to stop
stopping org.apache.spark.deploy.master.Master
kedar@myhost:~/software/spark$ ./sbin/start-all.sh 
starting org.apache.spark.deploy.master.Master, logging to /home/kedar/software/spark/logs/spark-kedar-org.apache.spark.deploy.master.Master-1-myhost.out
localhost: starting org.apache.spark.deploy.worker.Worker, logging to /home/kedar/software/spark/logs/spark-kedar-org.apache.spark.deploy.worker.Worker-1-myhost.out

问题是我的java替代方案指向了错误的java版本，下面是在ubuntu上解决问题的步骤，正如我们所看到的，Spark也成功启动了：

kedar@myhost:/opt$ sudo update-alternatives --install "/usr/bin/javac" "javac" "/opt/jdk1.8.0_72/bin/javac"
javac
kedar@myhost:/opt$ sudo update-alternatives --install "/usr/bin/javac" "javac" "/opt/jdk1.8.0_72/bin/javac" 1
kedar@myhost:/opt$ ll javac
ls: cannot access javac: No such file or directory
kedar@myhost:/opt$ sudo update-alternatives --config java
There is only one alternative in link group java (providing /usr/bin/java): /usr/lib/jvm/java-7-openjdk-amd64/jre/bin/java
Nothing to configure.
kedar@myhost:/opt$ sudo update-alternatives --config javac
There are 2 choices for the alternative javac (providing /usr/bin/javac).

  Selection    Path                                         Priority   Status
------------------------------------------------------------
* 0            /usr/lib/jvm/java-7-openjdk-amd64/bin/javac   1071      auto mode
  1            /opt/jdk1.8.0_72/bin/javac                    1         manual mode
  2            /usr/lib/jvm/java-7-openjdk-amd64/bin/javac   1071      manual mode

Press enter to keep the current choice[*], or type selection number: 1       
update-alternatives: using /opt/jdk1.8.0_72/bin/javac to provide /usr/bin/javac (javac) in manual mode
kedar@myhost:/opt$ sudo update-alternatives --config javac
There are 2 choices for the alternative javac (providing /usr/bin/javac).

  Selection    Path                                         Priority   Status
------------------------------------------------------------
  0            /usr/lib/jvm/java-7-openjdk-amd64/bin/javac   1071      auto mode
* 1            /opt/jdk1.8.0_72/bin/javac                    1         manual mode
  2            /usr/lib/jvm/java-7-openjdk-amd64/bin/javac   1071      manual mode

Press enter to keep the current choice[*], or type selection number: kedar@myhost:/opt$ 
kedar@myhost:/opt$ 
kedar@myhost:/opt$ 
kedar@myhost:/opt$ sudo update-alternatives --install "/usr/bin/java" "java" "/opt/jdk1.8.0_72/bin/java" 1
kedar@myhost:/opt$ ll javac
ls: cannot access javac: No such file or directory
kedar@myhost:/opt$ sudo update-alternatives --config java
There are 2 choices for the alternative java (providing /usr/bin/java).

  Selection    Path                                            Priority   Status
------------------------------------------------------------
* 0            /usr/lib/jvm/java-7-openjdk-amd64/jre/bin/java   1071      auto mode
  1            /opt/jdk1.8.0_72/bin/java                        1         manual mode
  2            /usr/lib/jvm/java-7-openjdk-amd64/jre/bin/java   1071      manual mode

Press enter to keep the current choice[*], or type selection number: 1
update-alternatives: using /opt/jdk1.8.0_72/bin/java to provide /usr/bin/java (java) in manual mode
kedar@myhost:/opt$ cd
kedar@myhost:~$ cd $SPARK_HOME
kedar@myhost:~/software/spark$ ./sbin/stop-all.sh 
localhost: no org.apache.spark.deploy.worker.Worker to stop
stopping org.apache.spark.deploy.master.Master
kedar@myhost:~/software/spark$ ./sbin/start-all.sh 
starting org.apache.spark.deploy.master.Master, logging to /home/kedar/software/spark/logs/spark-kedar-org.apache.spark.deploy.master.Master-1-myhost.out
localhost: starting org.apache.spark.deploy.worker.Worker, logging to /home/kedar/software/spark/logs/spark-kedar-org.apache.spark.deploy.worker.Worker-1-myhost.out

我看不到与java相关的问题，因为Master可以启动。Master和Worker是不同的机器，所以它可以启动。这似乎是多版本java问题，当我进一步调试它时，我看到我的java备选方案指向错误的java版本，我将发布正确的步骤来解决此问题。我看不到与java相关的问题，因为Master可以启动。Master和Worker是不同的机器，所以它可以启动。这似乎是多版本java的问题，当我进一步调试它时，我看到我的java替代品指向错误的java版本，我将发布解决此问题的正确步骤。