Mapreduce 在Hive Tez上映射减少日志
在HiveTez上运行查询后,我想获得MapReduce日志的解释?信息:后面的行传达了什么? 这里我附上了一个样品Mapreduce 在Hive Tez上映射减少日志,mapreduce,hive,yarn,hiveql,apache-tez,Mapreduce,Hive,Yarn,Hiveql,Apache Tez,在HiveTez上运行查询后,我想获得MapReduce日志的解释?信息:后面的行传达了什么? 这里我附上了一个样品 INFO : Session is already open INFO : Dag name: SELECT a.Model...) INFO : Tez session was closed. Reopening... INFO : Session re-established. INFO : INFO : Status: Running (Executing o
INFO : Session is already open
INFO : Dag name: SELECT a.Model...)
INFO : Tez session was closed. Reopening...
INFO : Session re-established.
INFO :
INFO : Status: Running (Executing on YARN cluster with App id application_14708112341234_1234)
INFO : Map 1: -/- Map 3: -/- Map 4: -/- Map 7: -/- Reducer 2: 0/15 Reducer 5: 0/26 Reducer 6: 0/13
INFO : Map 1: -/- Map 3: 0/118 Map 4: 0/118 Map 7: 0/1 Reducer 2: 0/15 Reducer 5: 0/26 Reducer 6: 0/13
INFO : Map 1: 0/118 Map 3: 0/118 Map 4: 0/118 Map 7: 0/1 Reducer 2: 0/15 Reducer 5: 0/26 Reducer 6: 0/13
INFO : Map 1: 0/118 Map 3: 0/118 Map 4: 0(+5)/118 Map 7: 0/1 Reducer 2: 0/15 Reducer 5: 0/26 Reducer 6: 0/13
INFO : Map 1: 0/118 Map 3: 0(+5)/118 Map 4: 0(+7)/118 Map 7: 0(+1)/1 Reducer 2: 0/15 Reducer 5: 0/26 Reducer 6: 0/13
INFO : Map 1: 0/118 Map 3: 0(+15)/118 Map 4: 0(+18)/118 Map 7: 0(+1)/1 Reducer 2: 0/15 Reducer 5: 0/26 Reducer 6: 0/13
您发布的日志是DAG执行日志。DAG由以下部分组成: 贴图1、贴图3、贴图4、贴图7贴图器顶点和还原器:还原器2、还原器5、还原器6
贴图1:-/-
-这意味着顶点尚未初始化,贴图器的数量尚未计算
映射4:0(+7)/118
-这意味着总共有118个映射器,其中7个正在并行运行,0个已完成,118-7=111个待定
减速器2:0/15
-这意味着总共有15个减速器,其中0个正在运行,0个已完成(15个减速器待定)
负数(在您的示例中没有)=失败或终止的映射程序或还原程序的数量
Qubole对Tez log有解释:正是我想要的!你有消息来源吗?我没有消息来源。但是,在执行tez期间,通过查看作业跟踪器,您可以很容易地检查这一点。@Dimpl在Qubole上发现了这一点: