Warning: file_get_contents(/data/phpspider/zhask/data//catemap/0/hadoop/6.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Hadoop “信息:Tez会议还没有建立。开幕式会议”挂起_Hadoop_Mapreduce_Hive_Tez - Fatal编程技术网

Hadoop “信息:Tez会议还没有建立。开幕式会议”挂起

Hadoop “信息:Tez会议还没有建立。开幕式会议”挂起,hadoop,mapreduce,hive,tez,Hadoop,Mapreduce,Hive,Tez,我正在使用HDP2.3.0.0-2557,并尝试运行如下配置单元查询: beeline> !connect jdbc:hive2://localhost:10000 Connecting to jdbc:hive2://localhost:10000 Enter username for jdbc:hive2://localhost:10000: root Enter password for jdbc:hive2://localhost:10000: ****** Connected t

我正在使用HDP2.3.0.0-2557,并尝试运行如下配置单元查询:

beeline> !connect jdbc:hive2://localhost:10000
Connecting to jdbc:hive2://localhost:10000
Enter username for jdbc:hive2://localhost:10000: root
Enter password for jdbc:hive2://localhost:10000: ******
Connected to: Apache Hive (version 1.2.1.2.3.0.0-2557)
Driver: Hive JDBC (version 1.2.1.2.3.0.0-2557)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://localhost:10000> select * from iitest1;
+----------------+----------------+----------------+----------------+----------------+----------------+-------------------------------------------------------------------------------------------------------+--+
| iitest1.it1c1  | iitest1.it1c2  | iitest1.it1c3  | iitest1.it1c4  | iitest1.it1c5  | iitest1.it1c6   |                                             iitest1.it1c7                                             |
+----------------+----------------+----------------+----------------+----------------+----------------+-------------------------------------------------------------------------------------------------------+--+
| 10             | 84000          | A              | A              | Jonathan       | Liu            | DGGBNEKBRXLBGKTPAGNPEBOQPIPDUAJLTLMRATQPKFPULKKXKGFFWHGOXFFJFLIFKITHKUQSPNFLQKFUIWHXLSTHPBKLMNUSQWOB  |
| 56             | 46000          | F              | F              | 12345          | 12345          | GIIWEVNSPPRLRDKWIXAUFHVNNJISHUWQADQMLWDBRWKWFSXDDXBGDVFRAWEOBTXXRTMMMAOIQUSQBEUGMQKNHFEBFUQECDAOWJMN  |
| 45             | 77000          | K              | K              | Lucy           | King           | HJMTHFFEQBSSCHWSGKNHIQFNKMQFNQHQKNTGSRLVFFJXOAFTDARTMVOCWFIAMICLAVCJBWVKQWAPLBEXMLDGGSVVDJXBDTFLQORQ  |
| NULL           | 77000          | K              | K              | Lucy           | King           | HJMTHFFEQBSSCHWSGKNHIQFNKMQFNQHQKNTGSRLVFFJXOAFTDARTMVOCWFIAMICLAVCJBWVKQWAPLBEXMLDGGSVVDJXBDTFLQORQ  |
+----------------+----------------+----------------+----------------+----------------+----------------+-------------------------------------------------------------------------------------------------------+--+
4 rows selected (8.578 seconds)
0: jdbc:hive2://localhost:10000> select max(it1c1) from iitest1;
INFO  : Tez session hasn't been created yet. Opening session
在接下来的比赛中,没有任何表现,而且似乎是特兹出了什么问题

此外,我尝试使用mapreduce而不是tez运行相同的查询,mapreduce作业挂起以及使用hive-hiveconf hive.execution.engine=mr启动hive CLI

我想知道是否有人能就如何解决这个问题提供一些建议

多谢


编辑1:我发现下面的链接,似乎没有足够的资源给tez。所以,我想知道如何清理被占用的资源,以便我有足够的资源给tez。

我也遇到了类似的问题。这可能是另一个阻止你的Tez会话运行的过程——在我的例子中,是Spark Thrift服务器阻止了它。监视ResourceManager节点上的内容Web UI默认端口为8088,请参阅页面底部。一旦旧货店的服务器被杀了,它就像一个魔咒一样运行着

我也遇到过类似的问题。这可能是另一个阻止你的Tez会话运行的过程——在我的例子中,是Spark Thrift服务器阻止了它。监视ResourceManager节点上的内容Web UI默认端口为8088,请参阅页面底部。一旦旧货店的服务器被杀了,它就像一个魔咒一样运行着

我和你有同样的问题

我通过修改以下内容解决了这个问题

reduce-Tez-config:Tez.am.resource.memory.mb=256

默认值为2048

并增加纱线:为节点上的所有纱线容器分配的内存


我猜在HiveCLI上使用Tez需要{Tez.am.resource.memory.mb}内存。当{为节点上的所有纱线容器分配的内存}没有足够的内存时。然后应用程序id被挂起。

我和您有同样的问题

我通过修改以下内容解决了这个问题

reduce-Tez-config:Tez.am.resource.memory.mb=256

默认值为2048

并增加纱线:为节点上的所有纱线容器分配的内存

我猜在HiveCLI上使用Tez需要{Tez.am.resource.memory.mb}内存。当{为节点上的所有纱线容器分配的内存}没有足够的内存时。然后应用程序id被挂起

hive> select max(IT1C1+IT1C2) from iitest1;
Query ID = root_20160713172448_09ccc9e6-c70d-4e56-b428-e511f62db9a6
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapreduce.job.reduces=<number>
Starting Job = job_1468423564920_0003, Tracking URL = http://sandbox.hortonworks.com:8088/proxy/application_1468423564920_0003/
Kill Command = /usr/hdp/2.3.0.0-2557/hadoop/bin/hadoop job  -kill job_1468423564920_0003
Interrupting... Be patient, this might take some time.
Press Ctrl+C again to kill JVM
killing job with: job_1468423564920_0003
Hadoop job information for Stage-1: number of mappers: 0; number of reducers: 0
2016-07-13 17:49:56,510 Stage-1 map = 0%,  reduce = 0%
Ended Job = job_1468423564920_0003 with errors
Error during job, obtaining debugging information...
FAILED: Execution Error, return code 2 from  org.apache.hadoop.hive.ql.exec.mr.MapRedTask
MapReduce Jobs Launched: 
Stage-Stage-1:  HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec
hive> exit
    > ;