Warning: file_get_contents(/data/phpspider/zhask/data//catemap/3/apache-spark/5.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Hadoop 启动apachespark集群_Hadoop_Apache Spark_Configuration_Installation - Fatal编程技术网

Hadoop 启动apachespark集群

Hadoop 启动apachespark集群,hadoop,apache-spark,configuration,installation,Hadoop,Apache Spark,Configuration,Installation,我已经在我的四节点集群上安装了Hadoop。我还为每一台安装了ApacheSpark。我可以在没有密码的情况下从主服务器ssh到每个从服务器。我也可以很好地启动主节点。但是,当我尝试使用/opt/Spark/sbin/start-all.sh运行Spark时,会出现以下错误: starting org.apache.spark.deploy.master.Master, logging to /opt/spark/logs/spark-hduser-org.apache.spark.deploy

我已经在我的四节点集群上安装了Hadoop。我还为每一台安装了ApacheSpark。我可以在没有密码的情况下从主服务器ssh到每个从服务器。我也可以很好地启动主节点。但是,当我尝试使用/opt/Spark/sbin/start-all.sh运行Spark时,会出现以下错误:

starting org.apache.spark.deploy.master.Master, logging to /opt/spark/logs/spark-hduser-org.apache.spark.deploy.master.Master-1-lebron.out
doublet: chown: changing ownership of ‘/opt/spark/logs’: Operation not permitted
doublet: starting org.apache.spark.deploy.worker.Worker, logging to /opt/spark/logs/spark-hduser-org.apache.spark.deploy.worker.Worker-1-doublet.out
doublet: /opt/spark/sbin/spark-daemon.sh: line 149: /opt/spark/logs/spark-hduser-org.apache.spark.deploy.worker.Worker-1-doublet.out: Permission denied
kyrie: chown: changing ownership of ‘/opt/spark/logs’: Operation not permitted
kyrie: starting org.apache.spark.deploy.worker.Worker, logging to /opt/spark/logs/spark-hduser-org.apache.spark.deploy.worker.Worker-1-kyrie.out
kyrie: /opt/spark/sbin/spark-daemon.sh: line 149: /opt/spark/logs/spark-hduser-org.apache.spark.deploy.worker.Worker-1-kyrie.out: Permission denied
lebron: starting org.apache.spark.deploy.worker.Worker, logging to /opt/spark/logs/spark-hduser-org.apache.spark.deploy.worker.Worker-1-lebron.out
jr: chown: changing ownership of ‘/opt/spark/logs’: Operation not permitted
jr: starting org.apache.spark.deploy.worker.Worker, logging to /opt/spark/logs/spark-hduser-org.apache.spark.deploy.worker.Worker-1-jr.out
jr: /opt/spark/sbin/spark-daemon.sh: line 149: /opt/spark/logs/spark-hduser-org.apache.spark.deploy.worker.Worker-1-jr.out: Permission denied
doublet: failed to launch org.apache.spark.deploy.worker.Worker:
doublet: tail: cannot open ‘/opt/spark/logs/spark-hduser-org.apache.spark.deploy.worker.Worker-1-doublet.out’ for reading: No such file or directory
doublet: full log in /opt/spark/logs/spark-hduser-org.apache.spark.deploy.worker.Worker-1-doublet.out
kyrie: failed to launch org.apache.spark.deploy.worker.Worker:
kyrie: tail: cannot open ‘/opt/spark/logs/spark-hduser-org.apache.spark.deploy.worker.Worker-1-kyrie.out’ for reading: No such file or directory
jr: failed to launch org.apache.spark.deploy.worker.Worker:
kyrie: full log in /opt/spark/logs/spark-hduser-org.apache.spark.deploy.worker.Worker-1-kyrie.out
jr: tail: cannot open ‘/opt/spark/logs/spark-hduser-org.apache.spark.deploy.worker.Worker-1-jr.out’ for reading: No such file or directory
jr: full log in /opt/spark/logs/spark-hduser-org.apache.spark.deploy.worker.Worker-1-jr.out
<我的电脑叫勒布朗(大师),Kyre,Jr,Mouple(工人)


请帮忙

在所有节点上配置无密码ssh

看起来/opt/spark文件夹没有写入日志文件的权限 给予完全许可:- sudo chmod-R 777/opt/spark 还要检查/opt/spark folder user:group对于所有机器(主设备和从设备)是否具有相同的值,否则请运行此命令 sudo chown-R用户名:groupName/opt/spark启动服务器:

要启动独立主服务器,请执行以下操作:

$ ./sbin/start-master.sh
要启动一个或多个辅助进程并通过以下方式将其连接到主进程:

$ ./sbin/start-slave.sh <master-spark-URL>
$。/sbin/start-slave.sh
看看这篇文章: