Amazon ec2 在EC2上部署spark
我在EC2上部署TwitterStreaming-0.0.1-SNAPSHOT时出错 我用了这个:Amazon ec2 在EC2上部署spark,amazon-ec2,apache-spark,spark-streaming,Amazon Ec2,Apache Spark,Spark Streaming,我在EC2上部署TwitterStreaming-0.0.1-SNAPSHOT时出错 我用了这个: spark/bin/spark-submit --class streaming.java.TwitterStreaming --master spark://xxxxxxxx.us-west-2.compute.amazonaws.com:6066 --deploy-mode cluster application-jar TwitterStreaming-0.0.1-SNAPSHOT.jar
spark/bin/spark-submit --class streaming.java.TwitterStreaming --master spark://xxxxxxxx.us-west-2.compute.amazonaws.com:6066 --deploy-mode cluster application-jar TwitterStreaming-0.0.1-SNAPSHOT.jar
答复是:
{
"action" : "CreateSubmissionResponse",
"message" : "Driver successfully submitted as driver-20150402200153-0000",
"serverSparkVersion" : "1.3.0",
"submissionId" : "driver-20150402200153-0000",
"success" : true
}
在浏览器上,我发现以下错误:
driver-20150402210100-0001 streaming.java.TwitterStreaming ERROR 1 512.0 MB stdout stderr java.io.FileNotFoundException: File file:/root/application-jar does not exist.
submit命令中的应用程序jar似乎是额外的
spark/bin/spark-submit --class streaming.java.TwitterStreaming --master spark://xxxxxxxx.us-west-2.compute.amazonaws.com:6066 --deploy-mode cluster TwitterStreaming-0.0.1-SNAPSHOT.jar
确保提供TwitterStreaming-0.0.1-SNAPSHOT.jar的完整路径始终错误:java.io.FileNotFoundException:File File File:/root/TwitterStreaming-0.0.1-SNAPSHOT.jar不存在。