Warning: file_get_contents(/data/phpspider/zhask/data//catemap/2/python/291.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181

Warning: file_get_contents(/data/phpspider/zhask/data//catemap/4/powerbi/2.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
`使用Thrift在Python脚本中运行配置单元查询时,连接被拒绝_Python_Hadoop_Mapreduce_Hive_Thrift - Fatal编程技术网

`使用Thrift在Python脚本中运行配置单元查询时,连接被拒绝

`使用Thrift在Python脚本中运行配置单元查询时,连接被拒绝,python,hadoop,mapreduce,hive,thrift,Python,Hadoop,Mapreduce,Hive,Thrift,全部, 我正在尝试使用Thrift library for python在python脚本中运行配置单元查询。 我能够运行不执行M/R的查询,如创建表,以及从表中选择*等。 但当我执行执行M/R作业的查询时(如select*fromtable where…),我会得到以下异常 starting hive server... Hive history file=/tmp/root/hive_job_log_root_201212171354_275968533.txt Total MapRedu

全部,

我正在尝试使用Thrift library for python在python脚本中运行配置单元查询。 我能够运行不执行M/R的查询,如
创建表
,以及
从表中选择*等。
但当我执行执行M/R作业的查询时(如
select*fromtable where…
),我会得到以下异常

starting hive server...

Hive history file=/tmp/root/hive_job_log_root_201212171354_275968533.txt
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
java.net.ConnectException: Call to sp-rhel6-01/172.22.193.79:54311 failed on connection exception: java.net.ConnectException: Connection refused

Job Submission failed with exception 'java.net.ConnectException(Call to sp-rhel6-01/172.22.193.79:54311 failed on connection exception: java.net.ConnectException: Connection refused)'
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MapRedTask
有谁能帮我理解出了什么问题吗


-Sushant

我在完成
SELECT
查询时遇到问题,尽管我可以完成
SHOW
description
查询。我解决这个问题的方法是重新启动集群上的服务。
我正在使用Cloudera管理集群,因此我运行的命令是
$sudo/etc/init.d/Cloudera-scm-agent hard\u restart
。我没有花太多时间调试,但我猜NN或JT崩溃了。有趣的是,我仍然可以完成对元数据的查询。我最好的猜测是,查询直接进入元存储,而不必触及HDFS。不过,我需要有人确认。

执行脚本时,
jps
的输出是什么?我能够解决这个问题。出现群集设置问题。
from hive_service import ThriftHive
from thrift import Thrift
from thrift.transport import TSocket
from thrift.transport import TTransport
from thrift.protocol import TBinaryProtocol

transport = TSocket.TSocket('172.22.193.79', 10000)
transport = TTransport.TBufferedTransport(transport)
protocol = TBinaryProtocol.TBinaryProtocol(transport)

client = ThriftHive.Client(protocol)
transport.open()

client.execute("select count(*) from example ")
print client.fetchAll();
transport.close()