当我运行java程序提交spark应用程序时,我得到;正在重试连接到服务器:0.0.0.0/0.0.0.0:8032”;信息
这是我的Java代码:当我运行java程序提交spark应用程序时,我得到;正在重试连接到服务器:0.0.0.0/0.0.0.0:8032”;信息,java,hadoop,apache-spark,yarn,resourcemanager,Java,Hadoop,Apache Spark,Yarn,Resourcemanager,这是我的Java代码: import org.apache.spark.deploy.SparkSubmit; public class Test{ public static void main(String[] args) { String[] arg0=new String[]{ "--master","yarn", "--deploy-mode","client", "--class
import org.apache.spark.deploy.SparkSubmit;
public class Test{
public static void main(String[] args) {
String[] arg0=new String[]{
"--master","yarn",
"--deploy-mode","client",
"--class","org.apache.spark.examples.SparkPi",
"/opt/spark-1.6.2-bin-hadoop2.6/lib/spark-examples-1.6.2-hadoop2.6.0.jar"
};
SparkSubmit.main(arg0);
}
}
我编译代码如下:
javac -cp ".:/opt/spark-1.6.2-bin-hadoop2.6/lib/spark-assembly-1.6.2-hadoop2.6.0.jar" Test.java
export HADOOP_CONF_DIR=/opt/yarn_conf
然后我将环境变量设置为:
javac -cp ".:/opt/spark-1.6.2-bin-hadoop2.6/lib/spark-assembly-1.6.2-hadoop2.6.0.jar" Test.java
export HADOOP_CONF_DIR=/opt/yarn_conf
/opt/warn_conf是从纱线集群/opt/hadoop-2.6.5/etc/hadoop上的目录复制的/
/opt/warn_conf/warn-site.xml已配置以下选项
<property>
<name>yarn.resourcemanager.address</name>
<value>10.110.16.60:8032</value>
</property>
<property>
<name>yarn.resourcemanager.scheduler.address</name>
<value>10.110.16.60:8030</value>
</property>
<property>
<name>yarn.resourcemanager.resource-tracker.address</name>
<value>10.110.16.60:8031</value>
</property>
....
然后我得到了如下信息:
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
17/02/20 16:22:31 INFO SparkContext: Running Spark version 1.6.2
17/02/20 16:22:31 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
17/02/20 16:22:31 INFO SecurityManager: Changing view acls to: root
17/02/20 16:22:31 INFO SecurityManager: Changing modify acls to: root
17/02/20 16:22:31 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(root); users with modify permissions: Set(root)
17/02/20 16:22:32 INFO Utils: Successfully started service 'sparkDriver' on port 55141.
17/02/20 16:22:32 INFO Slf4jLogger: Slf4jLogger started
17/02/20 16:22:32 INFO Remoting: Starting remoting
17/02/20 16:22:32 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://sparkDriverActorSystem@10.110.16.61:37984]
17/02/20 16:22:32 INFO Utils: Successfully started service 'sparkDriverActorSystem' on port 37984.
17/02/20 16:22:32 INFO SparkEnv: Registering MapOutputTracker
17/02/20 16:22:32 INFO SparkEnv: Registering BlockManagerMaster
17/02/20 16:22:33 INFO DiskBlockManager: Created local directory at /tmp/blockmgr-109e88d2-ce55-498c-bc0d-78448688fcac
17/02/20 16:22:33 INFO MemoryStore: MemoryStore started with capacity 1088.3 MB
17/02/20 16:22:33 INFO SparkEnv: Registering OutputCommitCoordinator
17/02/20 16:22:33 INFO Utils: Successfully started service 'SparkUI' on port 4040.
17/02/20 16:22:33 INFO SparkUI: Started SparkUI at http://10.110.16.61:4040
17/02/20 16:22:33 INFO HttpFileServer: HTTP File server directory is /tmp/spark-2d7ccc96-215f-402a-b425-bf4308bf6a94/httpd-d6478226-76dc-4904-873d-bc9d85e1fe18
17/02/20 16:22:33 INFO HttpServer: Starting HTTP Server
17/02/20 16:22:33 INFO Utils: Successfully started service 'HTTP file server' on port 58022.
17/02/20 16:22:33 INFO SparkContext: Added JAR file:/opt/spark-1.6.2-bin-hadoop2.6/lib/spark-examples-1.6.2-hadoop2.6.0.jar at http://10.110.16.61:58022/jars/spark-examples-1.6.2-hadoop2.6.0.jar with timestamp 1487578953758
17/02/20 16:22:33 INFO RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
17/02/20 16:22:35 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
17/02/20 16:22:36 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
17/02/20 16:22:37 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
17/02/20 16:22:38 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
我怎样才能改变它?思考。不知何故,您正试图连接到
0.0.0
,该地址不是有效的目标IP地址。