Warning: file_get_contents(/data/phpspider/zhask/data//catemap/0/hadoop/6.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Hadoop Mapreduce日志-文件与HDFS_Hadoop - Fatal编程技术网

Hadoop Mapreduce日志-文件与HDFS

Hadoop Mapreduce日志-文件与HDFS,hadoop,Hadoop,我提交了一个mapreduce,这是我的输出 下面给出的文件和HDFS之间的区别是什么 16/01/07 21:49:58 INFO mapreduce.Job: Counters: 38 File System Counters FILE: Number of bytes read=4011012 FILE: Number of bytes written=8400605 FILE: Number of read operations=0

我提交了一个mapreduce,这是我的输出

下面给出的文件和HDFS之间的区别是什么

16/01/07 21:49:58 INFO mapreduce.Job: Counters: 38
    File System Counters
        FILE: Number of bytes read=4011012
        FILE: Number of bytes written=8400605
        FILE: Number of read operations=0
        FILE: Number of large read operations=0
        FILE: Number of write operations=0
        HDFS: Number of bytes read=11928267
        HDFS: Number of bytes written=883509
        HDFS: Number of read operations=37
        HDFS: Number of large read operations=0
        HDFS: Number of write operations=6

文件-给出对中间文件执行的I/O量,中间文件在映射和缩减阶段(排序和洗牌阶段)之间进行内部维护


HDFS—映射程序读取的数据量和reducer写入的数据量。

文件—给出在映射和reduce阶段(排序和洗牌阶段)之间内部维护的中间文件上执行的I/O量

HDFS—映射器读取的数据量和reducer写入的数据量