Warning: file_get_contents(/data/phpspider/zhask/data//catemap/2/python/355.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Python客户端支持在Amazon EMR上运行配置单元_Python_Hive_Boto_Elastic Map Reduce - Fatal编程技术网

Python客户端支持在Amazon EMR上运行配置单元

Python客户端支持在Amazon EMR上运行配置单元,python,hive,boto,elastic-map-reduce,Python,Hive,Boto,Elastic Map Reduce,我注意到mrjob和boto都不支持Python接口在amazonelasticmapreduce(EMR)上提交和运行配置单元作业。是否有其他Python客户端库支持在EMR上运行配置单元 使用boto,您可以执行以下操作: args1 = [u's3://us-east-1.elasticmapreduce/libs/hive/hive-script', u'--base-path', u's3://us-east-1.elasticmapreduce/l

我注意到mrjob和boto都不支持Python接口在amazonelasticmapreduce(EMR)上提交和运行配置单元作业。是否有其他Python客户端库支持在EMR上运行配置单元

使用boto,您可以执行以下操作:

args1 = [u's3://us-east-1.elasticmapreduce/libs/hive/hive-script',
         u'--base-path',
         u's3://us-east-1.elasticmapreduce/libs/hive/',
         u'--install-hive',
         u'--hive-versions',
         u'0.7']
args2 = [u's3://us-east-1.elasticmapreduce/libs/hive/hive-script',
         u'--base-path',
         u's3://us-east-1.elasticmapreduce/libs/hive/',
         u'--hive-versions',
         u'0.7',
         u'--run-hive-script',
         u'--args',
         u'-f',
         s3_query_file_uri]
steps = []
for name, args in zip(('Setup Hive','Run Hive Script'),(args1,args2)):
    step = JarStep(name,
                   's3://us-east-1.elasticmapreduce/libs/script-runner/script-runner.jar',
                   step_args=args,
                   #action_on_failure="CANCEL_AND_WAIT"
                   )
    #should be inside loop
    steps.append(step)
# Kick off the job
jobid = EmrConnection().run_jobflow(name, s3_log_uri,
                                   steps=steps,
                                   master_instance_type=master_instance_type,
                                   slave_instance_type=slave_instance_type,
                                   num_instances=num_instances,
                                   hadoop_version="0.20")

由于验证错误,我将终止emr。。有什么想法吗?