Mapreduce 在Hive Tez上映射减少日志

Mapreduce 在Hive Tez上映射减少日志,mapreduce,hive,yarn,hiveql,apache-tez,Mapreduce,Hive,Yarn,Hiveql,Apache Tez,在HiveTez上运行查询后,我想获得MapReduce日志的解释?信息:后面的行传达了什么? 这里我附上了一个样品 INFO : Session is already open INFO : Dag name: SELECT a.Model...) INFO : Tez session was closed. Reopening... INFO : Session re-established. INFO : INFO : Status: Running (Executing o

在HiveTez上运行查询后,我想获得MapReduce日志的解释?信息:后面的行传达了什么? 这里我附上了一个样品

INFO  : Session is already open
INFO  : Dag name: SELECT a.Model...)
INFO  : Tez session was closed. Reopening...
INFO  : Session re-established.
INFO  : 
INFO  : Status: Running (Executing on YARN cluster with App id application_14708112341234_1234)

INFO  : Map 1: -/-  Map 3: -/-  Map 4: -/-  Map 7: -/-  Reducer 2: 0/15     Reducer 5: 0/26 Reducer 6: 0/13 
INFO  : Map 1: -/-  Map 3: 0/118    Map 4: 0/118    Map 7: 0/1  Reducer 2: 0/15 Reducer 5: 0/26  Reducer 6: 0/13
INFO  : Map 1: 0/118    Map 3: 0/118    Map 4: 0/118    Map 7: 0/1  Reducer 2: 0/15 Reducer 5: 0/26 Reducer 6: 0/13 
INFO  : Map 1: 0/118    Map 3: 0/118    Map 4: 0(+5)/118    Map 7: 0/1  Reducer 2: 0/15 Reducer 5: 0/26 Reducer 6: 0/13 
INFO  : Map 1: 0/118    Map 3: 0(+5)/118    Map 4: 0(+7)/118    Map 7: 0(+1)/1  Reducer 2: 0/15 Reducer 5: 0/26 Reducer 6: 0/13 
INFO  : Map 1: 0/118    Map 3: 0(+15)/118   Map 4: 0(+18)/118   Map 7: 0(+1)/1  Reducer 2: 0/15 Reducer 5: 0/26 Reducer 6: 0/13 

您发布的日志是DAG执行日志。DAG由以下部分组成: 贴图1、贴图3、贴图4、贴图7贴图器顶点和还原器:还原器2、还原器5、还原器6

贴图1:-/-
-这意味着顶点尚未初始化,贴图器的数量尚未计算

映射4:0(+7)/118
-这意味着总共有118个映射器,其中7个正在并行运行,0个已完成,118-7=111个待定

减速器2:0/15
-这意味着总共有15个减速器,其中0个正在运行,0个已完成(15个减速器待定)

负数(在您的示例中没有)=失败或终止的映射程序或还原程序的数量


Qubole对Tez log有解释:

正是我想要的!你有消息来源吗?我没有消息来源。但是,在执行tez期间,通过查看作业跟踪器,您可以很容易地检查这一点。@Dimpl在Qubole上发现了这一点: