Warning: file_get_contents(/data/phpspider/zhask/data//catemap/3/apache-spark/6.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Apache spark dcos chronos spark向mesos主机提交命令失败,因为未经授权_Apache Spark_Mesos_Mesosphere_Dcos - Fatal编程技术网

Apache spark dcos chronos spark向mesos主机提交命令失败,因为未经授权

Apache spark dcos chronos spark向mesos主机提交命令失败,因为未经授权,apache-spark,mesos,mesosphere,dcos,Apache Spark,Mesos,Mesosphere,Dcos,我们正在使用RESTAPI提交spark作业,以在dcos集群中创建spark master/cluster manager 下面的工作很完美 rest端点=>service/spark/v1/submissions/create { "action" : "CreateSubmissionRequest", "appArgs" : [ "100" ], "appResource" : "https://<s3 location>/spark-examples-1.5.1-hadoo

我们正在使用RESTAPI提交spark作业,以在dcos集群中创建spark master/cluster manager

下面的工作很完美

rest端点=>service/spark/v1/submissions/create

{
"action" : "CreateSubmissionRequest",
"appArgs" : [ "100" ],
"appResource" : "https://<s3 location>/spark-examples-1.5.1-hadoop2.4.0.jar",
"clientSparkVersion" : "1.6.1",
"environmentVariables" : {
"SPARK_ENV_LOADED" : "1",
"SPARK_JAVA_OPTS" : "-Dspark.mesos.coarse=true -Dspark.mesos.executor.docker.image=mesosphere/spark:1.0.0-1.6.1-2"
},
"mainClass" : "org.apache.spark.examples.SparkPi",
"sparkProperties" : {
"spark.jars" : "https://<s3 location>/spark-examples-1.5.1-hadoop2.4.0.jar",
"spark.app.name" : "SparkPi",
"spark.submit.deployMode" : "cluster",
"spark.master" : "mesos://<dcos mesos master>/service/spark/",
"spark.executor.cores" : "1",
"spark.executor.memory" : "2048m",
"spark.cores.max" : "2",
"spark.mesos.executor.docker.image" : "mesosphere/spark:1.0.0-1.6.1-2"
}
}
{
“操作”:“CreateSubmissionRequest”,
“装置”:[“100”],
“通知来源”:https:///spark-examples-1.5.1-hadoop2.4.0.jar",
“clientSparkVersion”:“1.6.1”,
“环境变量”:{
“SPARK_ENV_LOADED”:“1”,
“SPARK_JAVA_OPTS”:“-Dspark.mesos.rough=true-Dspark.mesos.executor.docker.image=mesosphere/SPARK:1.0.0-1.6.1-2”
},
“mainClass”:“org.apache.spark.examples.SparkPi”,
“火花财产”:{
“火花罐”:https:///spark-examples-1.5.1-hadoop2.4.0.jar",
“spark.app.name”:“SparkPi”,
“spark.submit.deployMode”:“集群”,
“火花大师”:mesos:///service/spark/",
“spark.executor.cores”:“1”,
“spark.executor.memory”:“2048m”,
“spark.cores.max”:“2”,
“spark.mesos.executor.docker.image”:“中间层/spark:1.0.0-1.6.1-2”
}
}
使用授权头授权令牌=${token}

当我把它交给chronos的时候 rest端点-/service/chronos/scheduler/iso8601

{
  "schedule": "R10/2016-06-16T08:28:00Z/PT2H",
  "name": "sparkjavachronos",
  "container": {
    "type": "DOCKER",
    "image": "mesosphere/spark:1.0.0-1.6.1-2"
  },
  "cpus": "0.5",
  "mem": "1024",
  "command": "/opt/spark/dist/bin/spark-submit --class org.apache.spark.examples.SparkPi --master mesos://<dcos mesos-master>/service/spark/ --deploy-mode cluster --supervise --executor-memory 2g --total-executor-cores 1 https://<s3 location>/spark-examples-1.5.1-hadoop2.4.0.jar 100"
}
{
“时间表”:“R10/2016-06-16T08:28:00Z/PT2H”,
“名称”:“sparkjavachronos”,
“容器”:{
“类型”:“DOCKER”,
“图像”:“中间层/火花:1.0.0-1.6.1-2”
},
“CPU”:“0.5”,
“mem”:“1024”,
“命令”:“/opt/spark/dist/bin/spark submit--class org.apache.spark.examples.SparkPi--mastermesos:///service/spark/ --部署模式群集--监控--执行器内存2g--执行器核心总数1https:///spark-examples-1.5.1-hadoop2.4.0.jar 100"
}
chronos作业提交正常,授权头授权令牌=${token}

但当chronos执行该命令时,它会得到响应,表明该请求未经授权。是否有办法将令牌转发到命令


或者,在具有授权令牌设置的dcos中,与群集管理器对话的chronos命令如何提供令牌。

嘿,您解决了这个问题吗?