Hadoop 分析参数时出错,amazon aws emr

Hadoop 分析参数时出错,amazon aws emr,hadoop,amazon-s3,hive,amazon-emr,s3distcp,Hadoop,Amazon S3,Hive,Amazon Emr,S3distcp,我正在尝试创建一个step by Linux控制台: aws emr add-steps --cluster-id j-XXXXXXXXXX --steps Type=CUSTOM_JAR,Name="S3DistCp step",Jar=/home/hadoop/lib/emr-s3distcp-1.0.jar,\ Args=["--s3Endpoint,s3-eu-west-1.amazonaws.com","--src,s3://folder-name/logs/j-XXXXXXXXXX

我正在尝试创建一个step by Linux控制台:

aws emr add-steps --cluster-id j-XXXXXXXXXX --steps Type=CUSTOM_JAR,Name="S3DistCp step",Jar=/home/hadoop/lib/emr-s3distcp-1.0.jar,\ 
Args=["--s3Endpoint,s3-eu-west-1.amazonaws.com","--src,s3://folder-name/logs/j-XXXXXXXXXX/node/","--dest,hdfs:///output","--srcPattern,.*[a-zA-Z,]+"]
我跳转以下错误

分析参数“--steps”时出错:输入应为“,”,收到“+”

我怎么能修好它

我正在寻找一种解决方案,将多个文件上传到S3和S3DistCp,即Amazon EMR的Hive gather。还有别的办法吗

我还有一个问题: 现在我正在创建一个SSH隧道来连接到Hive,如何连接PHP


目前,我已经通过删除“src模式”解决了这个错误,但是给了我另一个错误,我包括下面的图片

这就是出现的错误

INFO Synchronously wait child process to complete : hadoop jar /var/lib/aws/emr/step-runner/hadoop- 
INFO waitProcessCompletion ended with exit code 1 : hadoop jar
/var/lib/aws/emr/step-runner/hadoop-
INFO total process run time: 2 seconds
2016-07-12T14:26:48.744Z INFO Step created jobs:
2016-07-12T14:26:48.744Z WARN Step failed with exitCode 1 and took 2 seconds

谢谢

尝试JSON配置

[
    {
        "Name":"S3DistCp step",
        "Args":["s3-dist-cp","--s3Endpoint=s3.amazonaws.com","--src=s3://mybucket/logs/j-3GYXXXXXX9IOJ/node/","--dest=hdfs:///output","--srcPattern=.*[a-zA-Z,]+"],
        "ActionOnFailure":"CONTINUE",
        "Type":"CUSTOM_JAR",
        "Jar":"command-runner.jar"        
    }
]
aws emr添加步骤——集群id j-3GYXXXXX9IOK——步骤file://./myStep.json


解析错误很奇怪,您是否尝试将选项放在json文件中,并使用json文件调用命令,以查看它是否有帮助Shello Frederic,我必须努力删除“src模式”,现在我遇到另一个错误,它与amazon中的指定不太一致(我上升到图像上方)