Warning: file_get_contents(/data/phpspider/zhask/data//catemap/2/python/298.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181

Warning: file_get_contents(/data/phpspider/zhask/data//catemap/7/arduino/2.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Python sys.stdout.write()在MRJOB映射器中的位置?_Python_Emr_Mrjob - Fatal编程技术网

Python sys.stdout.write()在MRJOB映射器中的位置?

Python sys.stdout.write()在MRJOB映射器中的位置?,python,emr,mrjob,Python,Emr,Mrjob,mrjob.conf runners: emr: aws_access_key_id: ** aws_secret_access_key: ** aws_region: us-east-1 aws_availability_zone: us-east-1a ec2_key_pair: scrapers2 ec2_key_pair_file: ~/arachnid.pem ec2_instance_type: c3.8xlarge

mrjob.conf

runners:
  emr:
    aws_access_key_id: **
    aws_secret_access_key: **
    aws_region: us-east-1
    aws_availability_zone: us-east-1a
    ec2_key_pair: scrapers2
    ec2_key_pair_file: ~/arachnid.pem
    ec2_instance_type: c3.8xlarge
    ec2_master_instance_type: c3.8xlarge
    num_ec2_instances: 3
    python_bin: python2.6
    interpreter: python2.6
    ami_version: 2.4.11
    iam_job_flow_role: EMR_DefaultRole
    jobconf: {"mapred.task.timeout": 600000, "mapred.output.direct.NativeS3FileSystem": false}
    base_tmp_dir: /tmp
    enable_emr_debugging: true
    cmdenv:
        TZ: America/New_York
    s3_log_uri: s3://mrjob-lists/tmp/logs/
    s3_scratch_uri: s3://mrjob-lists/tmp/
    output_dir: s3://mrjob-lists/output
    ssh_tunnel_is_open: true
    ssh_tunnel_to_job_tracker: true
我正在使用emr运行作业,我的映射器任务有:

print "test"
以及

sys.stdout.write("TEst")

但是,我在S3的标准输出文件中找不到这个输出。输出写在哪里?

Hadoop 1作业的映射程序stdout应该出现在S3日志中的
/task truments/job#########/trument#########stdout.gz

它们确实需要一段时间才能推送到S3。如果让集群保持运行,可以检查Hadoop JobTracker web界面,并确保它在作业执行后也在日志中本地显示