Python Amazon EMR群集步骤成功，但输出为空？

我有两个为amazon EMR cluster编写的脚本，在我的pc上，它可以工作并创建一个文件，但在我的s3存储桶中，在步骤完成后，我得到了3个部分文件和一个0字节的成功文件

这是mapper.py

#!/usr/bin/env python

import sys
import re

# Constants declaration

WINDOW = 10
OVERLAP = 4
START_POSITION = 0
END_POSITION = 0

# regular expressions

pattern = re.compile("[a-z]*", re.IGNORECASE)

a_to_f_pattern = re.compile("[a-f]", re.IGNORECASE)
g_to_l_pattern = re.compile("[g-l]", re.IGNORECASE)
m_to_r_pattern = re.compile("[m-r]", re.IGNORECASE)
s_to_z_pattern = re.compile("[s-z]", re.IGNORECASE)

# variables initialization

converted_word = ""
next_word = ""
new_character = ""
filename = ""
prev_filename = ""
i = 0

def convert(word):
    if a_to_f_pattern.match(word[0]):
        # print "found match!first pattern!", word[i], filename
        new_character = "A"
    elif g_to_l_pattern.match(word[0]):
        # print "found match!second pattern!", word[i], filename
        new_character = "C"
    elif m_to_r_pattern.match(word[0]):
        # print "found match!third pattern!", word[i], filename
        new_character = "G"
    elif s_to_z_pattern.match(word[0]):
        # print "found match!fourth pattern!", word[i], filename
        new_character = "T"
    return new_character

# Read pairs as lines of input from STDIN
for line in sys.stdin:

    line.strip()

    if ":" in line:
        filename, line = line.split(':',1)
        filename = filename.replace("source_text//", "")
        filename = filename.replace("suspicious_text//", "")

    # initialize prev_filename
    if prev_filename == "":
        prev_filename = filename

    # check if its a new file, and reset start position
    if filename != prev_filename:

        START_POSITION = 0
        next_word = ""
        converted_word = ""
        prev_filename = filename

    # loop through every word that matches the pattern
    for word in pattern.findall(line):
        while i < 1 and len(word) > 0:
            if len(converted_word) != WINDOW:

                new_character = convert(word)
                converted_word = converted_word + new_character

                if len(converted_word) > (WINDOW - OVERLAP):
                    next_word = next_word + new_character

                # print "word= ", word
                # print "converted_word= ", converted_word
            else:

                END_POSITION = START_POSITION + (len(converted_word) - 1)

                print converted_word + "," + str(filename) + "," + str(START_POSITION) + "," + str(END_POSITION)

                START_POSITION = START_POSITION + (WINDOW - OVERLAP)
                new_character = convert(word)
                converted_word = next_word + new_character
                # print "word= ", word
                # print "converted_word= ", converted_word
                next_word = ""
            # increment
            i = i + 1
        # reset value in order to be used for next word
        i = 0

这是EMR步骤的日志

2016-04-27 18:42:19,450 INFO com.amazon.ws.emr.hadoop.fs.EmrFileSystem (main): Consistency disabled, using com.amazon.ws.emr.hadoop.fs.s3n.S3NativeFileSystem as filesystem implementation
2016-04-27 18:42:19,694 INFO amazon.emr.metrics.MetricsSaver (main): MetricsConfigRecord disabledInCluster: false instanceEngineCycleSec: 60 clusterEngineCycleSec: 60 disableClusterEngine: true maxMemoryMb: 3072 maxInstanceCount: 500 lastModified: 1461781964683 
2016-04-27 18:42:19,694 INFO amazon.emr.metrics.MetricsSaver (main): Created MetricsSaver j-2IJ5EQ9RVEI01:i-364d48ee:RunJar:08615 period:60 /mnt/var/em/raw/i-364d48ee_20160427_RunJar_08615_raw.bin
2016-04-27 18:42:21,683 INFO org.apache.hadoop.yarn.client.RMProxy (main): Connecting to ResourceManager at ip-172-31-35-145.us-west-2.compute.internal/172.31.35.145:8032
2016-04-27 18:42:21,871 INFO org.apache.hadoop.yarn.client.RMProxy (main): Connecting to ResourceManager at ip-172-31-35-145.us-west-2.compute.internal/172.31.35.145:8032
2016-04-27 18:42:22,434 INFO com.amazon.ws.emr.hadoop.fs.s3n.S3NativeFileSystem (main): Opening 's3://source123/mapper.py' for reading
2016-04-27 18:42:22,553 INFO amazon.emr.metrics.MetricsSaver (main): Thread 1 created MetricsLockFreeSaver 1
2016-04-27 18:42:22,740 INFO com.amazon.ws.emr.hadoop.fs.s3n.S3NativeFileSystem (main): Opening 's3://source123/source_reducer.py' for reading
2016-04-27 18:42:22,936 INFO com.hadoop.compression.lzo.GPLNativeCodeLoader (main): Loaded native gpl library
2016-04-27 18:42:22,939 INFO com.hadoop.compression.lzo.LzoCodec (main): Successfully loaded & initialized native-lzo library [hadoop-lzo rev 426d94a07125cf9447bb0c2b336cf10b4c254375]
2016-04-27 18:42:23,285 INFO com.amazon.ws.emr.hadoop.fs.s3n.S3NativeFileSystem (main): listStatus s3://source123/input with recursive false
2016-04-27 18:42:23,505 INFO org.apache.hadoop.mapred.FileInputFormat (main): Total input paths to process : 1
2016-04-27 18:42:23,601 INFO org.apache.hadoop.mapreduce.JobSubmitter (main): number of splits:9
2016-04-27 18:42:25,047 INFO org.apache.hadoop.mapreduce.JobSubmitter (main): Submitting tokens for job: job_1461781956121_0002
2016-04-27 18:42:25,361 INFO org.apache.hadoop.yarn.client.api.impl.YarnClientImpl (main): Submitted application application_1461781956121_0002
2016-04-27 18:42:25,407 INFO org.apache.hadoop.mapreduce.Job (main): The url to track the job: http://ip-172-31-35-145.us-west-2.compute.internal:20888/proxy/application_1461781956121_0002/
2016-04-27 18:42:25,408 INFO org.apache.hadoop.mapreduce.Job (main): Running job: job_1461781956121_0002
2016-04-27 18:42:34,512 INFO org.apache.hadoop.mapreduce.Job (main): Job job_1461781956121_0002 running in uber mode : false
2016-04-27 18:42:34,514 INFO org.apache.hadoop.mapreduce.Job (main):  map 0% reduce 0%
2016-04-27 18:42:51,654 INFO org.apache.hadoop.mapreduce.Job (main):  map 11% reduce 0%
2016-04-27 18:42:54,675 INFO org.apache.hadoop.mapreduce.Job (main):  map 22% reduce 0%
2016-04-27 18:42:56,689 INFO org.apache.hadoop.mapreduce.Job (main):  map 33% reduce 0%
2016-04-27 18:42:59,708 INFO org.apache.hadoop.mapreduce.Job (main):  map 56% reduce 0%
2016-04-27 18:43:00,714 INFO org.apache.hadoop.mapreduce.Job (main):  map 67% reduce 0%
2016-04-27 18:43:08,760 INFO org.apache.hadoop.mapreduce.Job (main):  map 89% reduce 0%
2016-04-27 18:43:10,771 INFO org.apache.hadoop.mapreduce.Job (main):  map 100% reduce 0%
2016-04-27 18:43:12,783 INFO org.apache.hadoop.mapreduce.Job (main):  map 100% reduce 33%
2016-04-27 18:43:16,805 INFO org.apache.hadoop.mapreduce.Job (main):  map 100% reduce 67%
2016-04-27 18:43:19,820 INFO org.apache.hadoop.mapreduce.Job (main):  map 100% reduce 100%
2016-04-27 18:43:20,833 INFO org.apache.hadoop.mapreduce.Job (main): Job job_1461781956121_0002 completed successfully
2016-04-27 18:43:20,982 INFO org.apache.hadoop.mapreduce.Job (main): Counters: 55
    File System Counters
        FILE: Number of bytes read=114
        FILE: Number of bytes written=1541394
        FILE: Number of read operations=0
        FILE: Number of large read operations=0
        FILE: Number of write operations=0
        HDFS: Number of bytes read=873
        HDFS: Number of bytes written=0
        HDFS: Number of read operations=9
        HDFS: Number of large read operations=0
        HDFS: Number of write operations=0
        S3: Number of bytes read=864
        S3: Number of bytes written=0
        S3: Number of read operations=0
        S3: Number of large read operations=0
        S3: Number of write operations=0
    Job Counters 
        Killed map tasks=1
        Launched map tasks=9
        Launched reduce tasks=3
        Data-local map tasks=9
        Total time spent by all maps in occupied slots (ms)=6477030
        Total time spent by all reduces in occupied slots (ms)=2225520
        Total time spent by all map tasks (ms)=143934
        Total time spent by all reduce tasks (ms)=24728
        Total vcore-milliseconds taken by all map tasks=143934
        Total vcore-milliseconds taken by all reduce tasks=24728
        Total megabyte-milliseconds taken by all map tasks=207264960
        Total megabyte-milliseconds taken by all reduce tasks=71216640
    Map-Reduce Framework
        Map input records=5
        Map output records=3
        Map output bytes=52
        Map output materialized bytes=489
        Input split bytes=873
        Combine input records=0
        Combine output records=0
        Reduce input groups=3
        Reduce shuffle bytes=489
        Reduce input records=3
        Reduce output records=0
        Spilled Records=6
        Shuffled Maps =27
        Failed Shuffles=0
        Merged Map outputs=27
        GC time elapsed (ms)=2100
        CPU time spent (ms)=11850
        Physical memory (bytes) snapshot=5267861504
        Virtual memory (bytes) snapshot=28408479744
        Total committed heap usage (bytes)=6102188032
    Shuffle Errors
        BAD_ID=0
        CONNECTION=0
        IO_ERROR=0
        WRONG_LENGTH=0
        WRONG_MAP=0
        WRONG_REDUCE=0
    File Input Format Counters 
        Bytes Read=864
    File Output Format Counters 
        Bytes Written=0
2016-04-27 18:43:20,983 INFO org.apache.hadoop.streaming.StreamJob (main): Output directory: s3://source123/output/

我做错什么了吗

2016-04-27 18:42:19,450 INFO com.amazon.ws.emr.hadoop.fs.EmrFileSystem (main): Consistency disabled, using com.amazon.ws.emr.hadoop.fs.s3n.S3NativeFileSystem as filesystem implementation
2016-04-27 18:42:19,694 INFO amazon.emr.metrics.MetricsSaver (main): MetricsConfigRecord disabledInCluster: false instanceEngineCycleSec: 60 clusterEngineCycleSec: 60 disableClusterEngine: true maxMemoryMb: 3072 maxInstanceCount: 500 lastModified: 1461781964683 
2016-04-27 18:42:19,694 INFO amazon.emr.metrics.MetricsSaver (main): Created MetricsSaver j-2IJ5EQ9RVEI01:i-364d48ee:RunJar:08615 period:60 /mnt/var/em/raw/i-364d48ee_20160427_RunJar_08615_raw.bin
2016-04-27 18:42:21,683 INFO org.apache.hadoop.yarn.client.RMProxy (main): Connecting to ResourceManager at ip-172-31-35-145.us-west-2.compute.internal/172.31.35.145:8032
2016-04-27 18:42:21,871 INFO org.apache.hadoop.yarn.client.RMProxy (main): Connecting to ResourceManager at ip-172-31-35-145.us-west-2.compute.internal/172.31.35.145:8032
2016-04-27 18:42:22,434 INFO com.amazon.ws.emr.hadoop.fs.s3n.S3NativeFileSystem (main): Opening 's3://source123/mapper.py' for reading
2016-04-27 18:42:22,553 INFO amazon.emr.metrics.MetricsSaver (main): Thread 1 created MetricsLockFreeSaver 1
2016-04-27 18:42:22,740 INFO com.amazon.ws.emr.hadoop.fs.s3n.S3NativeFileSystem (main): Opening 's3://source123/source_reducer.py' for reading
2016-04-27 18:42:22,936 INFO com.hadoop.compression.lzo.GPLNativeCodeLoader (main): Loaded native gpl library
2016-04-27 18:42:22,939 INFO com.hadoop.compression.lzo.LzoCodec (main): Successfully loaded & initialized native-lzo library [hadoop-lzo rev 426d94a07125cf9447bb0c2b336cf10b4c254375]
2016-04-27 18:42:23,285 INFO com.amazon.ws.emr.hadoop.fs.s3n.S3NativeFileSystem (main): listStatus s3://source123/input with recursive false
2016-04-27 18:42:23,505 INFO org.apache.hadoop.mapred.FileInputFormat (main): Total input paths to process : 1
2016-04-27 18:42:23,601 INFO org.apache.hadoop.mapreduce.JobSubmitter (main): number of splits:9
2016-04-27 18:42:25,047 INFO org.apache.hadoop.mapreduce.JobSubmitter (main): Submitting tokens for job: job_1461781956121_0002
2016-04-27 18:42:25,361 INFO org.apache.hadoop.yarn.client.api.impl.YarnClientImpl (main): Submitted application application_1461781956121_0002
2016-04-27 18:42:25,407 INFO org.apache.hadoop.mapreduce.Job (main): The url to track the job: http://ip-172-31-35-145.us-west-2.compute.internal:20888/proxy/application_1461781956121_0002/
2016-04-27 18:42:25,408 INFO org.apache.hadoop.mapreduce.Job (main): Running job: job_1461781956121_0002
2016-04-27 18:42:34,512 INFO org.apache.hadoop.mapreduce.Job (main): Job job_1461781956121_0002 running in uber mode : false
2016-04-27 18:42:34,514 INFO org.apache.hadoop.mapreduce.Job (main):  map 0% reduce 0%
2016-04-27 18:42:51,654 INFO org.apache.hadoop.mapreduce.Job (main):  map 11% reduce 0%
2016-04-27 18:42:54,675 INFO org.apache.hadoop.mapreduce.Job (main):  map 22% reduce 0%
2016-04-27 18:42:56,689 INFO org.apache.hadoop.mapreduce.Job (main):  map 33% reduce 0%
2016-04-27 18:42:59,708 INFO org.apache.hadoop.mapreduce.Job (main):  map 56% reduce 0%
2016-04-27 18:43:00,714 INFO org.apache.hadoop.mapreduce.Job (main):  map 67% reduce 0%
2016-04-27 18:43:08,760 INFO org.apache.hadoop.mapreduce.Job (main):  map 89% reduce 0%
2016-04-27 18:43:10,771 INFO org.apache.hadoop.mapreduce.Job (main):  map 100% reduce 0%
2016-04-27 18:43:12,783 INFO org.apache.hadoop.mapreduce.Job (main):  map 100% reduce 33%
2016-04-27 18:43:16,805 INFO org.apache.hadoop.mapreduce.Job (main):  map 100% reduce 67%
2016-04-27 18:43:19,820 INFO org.apache.hadoop.mapreduce.Job (main):  map 100% reduce 100%
2016-04-27 18:43:20,833 INFO org.apache.hadoop.mapreduce.Job (main): Job job_1461781956121_0002 completed successfully
2016-04-27 18:43:20,982 INFO org.apache.hadoop.mapreduce.Job (main): Counters: 55
    File System Counters
        FILE: Number of bytes read=114
        FILE: Number of bytes written=1541394
        FILE: Number of read operations=0
        FILE: Number of large read operations=0
        FILE: Number of write operations=0
        HDFS: Number of bytes read=873
        HDFS: Number of bytes written=0
        HDFS: Number of read operations=9
        HDFS: Number of large read operations=0
        HDFS: Number of write operations=0
        S3: Number of bytes read=864
        S3: Number of bytes written=0
        S3: Number of read operations=0
        S3: Number of large read operations=0
        S3: Number of write operations=0
    Job Counters 
        Killed map tasks=1
        Launched map tasks=9
        Launched reduce tasks=3
        Data-local map tasks=9
        Total time spent by all maps in occupied slots (ms)=6477030
        Total time spent by all reduces in occupied slots (ms)=2225520
        Total time spent by all map tasks (ms)=143934
        Total time spent by all reduce tasks (ms)=24728
        Total vcore-milliseconds taken by all map tasks=143934
        Total vcore-milliseconds taken by all reduce tasks=24728
        Total megabyte-milliseconds taken by all map tasks=207264960
        Total megabyte-milliseconds taken by all reduce tasks=71216640
    Map-Reduce Framework
        Map input records=5
        Map output records=3
        Map output bytes=52
        Map output materialized bytes=489
        Input split bytes=873
        Combine input records=0
        Combine output records=0
        Reduce input groups=3
        Reduce shuffle bytes=489
        Reduce input records=3
        Reduce output records=0
        Spilled Records=6
        Shuffled Maps =27
        Failed Shuffles=0
        Merged Map outputs=27
        GC time elapsed (ms)=2100
        CPU time spent (ms)=11850
        Physical memory (bytes) snapshot=5267861504
        Virtual memory (bytes) snapshot=28408479744
        Total committed heap usage (bytes)=6102188032
    Shuffle Errors
        BAD_ID=0
        CONNECTION=0
        IO_ERROR=0
        WRONG_LENGTH=0
        WRONG_MAP=0
        WRONG_REDUCE=0
    File Input Format Counters 
        Bytes Read=864
    File Output Format Counters 
        Bytes Written=0
2016-04-27 18:43:20,983 INFO org.apache.hadoop.streaming.StreamJob (main): Output directory: s3://source123/output/