Hadoop Oozie蜂巢动作悬挂，心脏永远跳动_Hadoop_Hive_Oozie

Hadoop Oozie蜂巢动作悬挂，心脏永远跳动

hadoop hive

Hadoop Oozie蜂巢动作悬挂，心脏永远跳动,hadoop,hive,oozie,Hadoop,Hive,Oozie,我试图通过我在Hue中创建的Oozie工作流运行配置单元操作，但是操作“心跳”永远不会停止，并且不会执行配置单元SQL 我读过其他关于心脏永远跳动的文章，但这篇文章似乎发生在另一个地方，在SQL语句被解析之后。我检查了集群中每个节点的内存，并验证了任务计数参数是否合理以下是hive-config.xml文件： <configuration> <property> <name>javax.jdo.option.ConnectionURL</n

我试图通过我在Hue中创建的Oozie工作流运行配置单元操作，但是操作“心跳”永远不会停止，并且不会执行配置单元SQL

我读过其他关于心脏永远跳动的文章，但这篇文章似乎发生在另一个地方，在SQL语句被解析之后。我检查了集群中每个节点的内存，并验证了任务计数参数是否合理

以下是hive-config.xml文件：

<configuration>

<property>   
  <name>javax.jdo.option.ConnectionURL</name>   
  <value>jdbc:hive://10.1.10.250:10000/testdb</value>   
  <description>JDBC connect string</description> 
</property>

<property>   
  <name>javax.jdo.option.ConnectionDriverName</name>   
  <value>org.apache.hadoop.hive.jdbc.HiveDriver</value>   
  <description>JDBC driver</description> 
</property>

</configuration>


javax.jdo.option.ConnectionURL
jdbc：hive://10.1.10.250:10000/testdb   
JDBC连接字符串
javax.jdo.option.ConnectionDriverName
org.apache.hadoop.hive.jdbc.HiveDriver
JDBC驱动程序

我知道配置单元连接正在工作，因为如果提供了错误的SQL语句、错误的URL或错误的驱动程序名称，则操作将失败

以下是操作标准日志：

  [...truncated]

  =================================================================

  >>> Invoking Hive command line now >>>

  4283 [main] INFO  org.apache.hadoop.hive.ql.log.PerfLogger  - <PERFLOG method=Driver.run from=org.apache.hadoop.hive.ql.Driver>
  4284 [main] INFO  org.apache.hadoop.hive.ql.log.PerfLogger  - <PERFLOG method=TimeToSubmit from=org.apache.hadoop.hive.ql.Driver>
  4284 [main] INFO  org.apache.hadoop.hive.ql.log.PerfLogger  - <PERFLOG method=compile from=org.apache.hadoop.hive.ql.Driver>
  4339 [main] INFO  org.apache.hadoop.hive.ql.log.PerfLogger  - <PERFLOG method=parse from=org.apache.hadoop.hive.ql.Driver>
  4354 [main] INFO  hive.ql.parse.ParseDriver  - Parsing command: create table testdb.temp99 (col1 int)
  4665 [main] INFO  hive.ql.parse.ParseDriver  - Parse Completed
  4667 [main] INFO  org.apache.hadoop.hive.ql.log.PerfLogger  - </PERFLOG method=parse start=1418968298270 end=1418968298598 duration=328 from=org.apache.hadoop.hive.ql.Driver>
  4667 [main] INFO  org.apache.hadoop.hive.ql.log.PerfLogger  - <PERFLOG method=semanticAnalyze from=org.apache.hadoop.hive.ql.Driver>
  4733 [main] INFO  org.apache.hadoop.hive.ql.parse.SemanticAnalyzer  - Starting Semantic Analysis
  4735 [main] INFO  org.apache.hadoop.hive.ql.parse.SemanticAnalyzer  - Creating table testdb.temp99 position=13
  4760 [main] INFO  org.apache.hadoop.hive.ql.Driver  - Semantic Analysis Completed
  4775 [main] INFO  org.apache.hadoop.hive.ql.log.PerfLogger  - </PERFLOG method=semanticAnalyze start=1418968298598 end=1418968298706 duration=108 from=org.apache.hadoop.hive.ql.Driver>
  4784 [main] INFO  org.apache.hadoop.hive.ql.Driver  - Returning Hive schema: Schema(fieldSchemas:null, properties:null)
  4784 [main] INFO  org.apache.hadoop.hive.ql.log.PerfLogger  - </PERFLOG method=compile start=1418968298215 end=1418968298715 duration=500 from=org.apache.hadoop.hive.ql.Driver>
  4785 [main] INFO  org.apache.hadoop.hive.ql.Driver  - Concurrency mode is disabled, not creating a lock manager
  4785 [main] INFO  org.apache.hadoop.hive.ql.log.PerfLogger  - <PERFLOG method=Driver.execute from=org.apache.hadoop.hive.ql.Driver>
  4785 [main] INFO  org.apache.hadoop.hive.ql.Driver  - Starting command: create table testdb.temp99 (col1 int)
  4792 [main] INFO  org.apache.hadoop.hive.ql.log.PerfLogger  - </PERFLOG method=TimeToSubmit start=1418968298215 end=1418968298723 duration=508 from=org.apache.hadoop.hive.ql.Driver>
  4792 [main] INFO  org.apache.hadoop.hive.ql.log.PerfLogger  - <PERFLOG method=runTasks from=org.apache.hadoop.hive.ql.Driver>
  4792 [main] INFO  org.apache.hadoop.hive.ql.log.PerfLogger  - <PERFLOG method=task.DDL.Stage-0 from=org.apache.hadoop.hive.ql.Driver>
  4815 [main] INFO  hive.ql.exec.DDLTask  - Default to LazySimpleSerDe for table testdb.temp99
  4935 [main] INFO  org.apache.hadoop.hive.metastore.HiveMetaStore  - 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
  4959 [main] INFO  org.apache.hadoop.hive.metastore.ObjectStore  - ObjectStore, initialize called
  5261 [main] INFO  DataNucleus.Persistence  - Property datanucleus.cache.level2 unknown - will be ignored
  Heart beat
  Heart beat
  [...forever...]

[…被截断]
=================================================================
>>>立即调用配置单元命令行>>>
4283[main]INFO org.apache.hadoop.hive.ql.log.PerfLogger-
4284[main]INFO org.apache.hadoop.hive.ql.log.PerfLogger-
4284[main]INFO org.apache.hadoop.hive.ql.log.PerfLogger-
4339[main]INFO org.apache.hadoop.hive.ql.log.PerfLogger-
4354[main]INFO hive.ql.parse.ParseDriver-解析命令：create table testdb.temp99（col1 int）
4665[main]INFO hive.ql.parse.ParseDriver-解析完成
4667[main]INFO org.apache.hadoop.hive.ql.log.PerfLogger-
4667[main]INFO org.apache.hadoop.hive.ql.log.PerfLogger-
4733[main]INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer-开始语义分析
4735[main]INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer-创建表testdb.temp99 position=13
4760[main]INFO org.apache.hadoop.hive.ql.Driver-语义分析已完成
4775[main]INFO org.apache.hadoop.hive.ql.log.PerfLogger-
4784[main]INFO org.apache.hadoop.hive.ql.Driver-返回配置单元模式：schema（fieldschema:null，properties:null）
4784[main]INFO org.apache.hadoop.hive.ql.log.PerfLogger-
4785[main]INFO org.apache.hadoop.hive.ql.Driver-禁用并发模式，不创建锁管理器
4785[main]INFO org.apache.hadoop.hive.ql.log.PerfLogger-
4785[main]INFO org.apache.hadoop.hive.ql.Driver-启动命令：create table testdb.temp99（col1 int）
4792[main]INFO org.apache.hadoop.hive.ql.log.PerfLogger-
4792[main]INFO org.apache.hadoop.hive.ql.log.PerfLogger-
4792[main]INFO org.apache.hadoop.hive.ql.log.PerfLogger-
4815[main]INFO hive.ql.exec.ddlstask-表testdb.temp99的默认值为LazySimpleSerDe
4935[main]INFO org.apache.hadoop.hive.metastore.HiveMetaStore-0:使用实现类打开原始存储：org.apache.hadoop.hive.metastore.ObjectStore
4959[main]INFO org.apache.hadoop.hive.metastore.ObjectStore-ObjectStore，初始化调用
5261[main]信息DataNucleus.Persistence-属性DataNucleus.cache.level2未知-将被忽略
心跳
心跳
[…永远…]

为什么工作流心脏在日志中的这一点上跳动，而不是继续

增编：

与此配置单元操作关联的Oozie工作流是：

<workflow-app name="Hive-copy" xmlns="uri:oozie:workflow:0.4">
    <start to="Hive"/>
    <action name="Hive">
        <hive xmlns="uri:oozie:hive-action:0.2">
            <job-tracker>${jobTracker}</job-tracker>
            <name-node>${nameNode}</name-node>
              <job-xml>/user/test/hive-config.xml</job-xml>
            <script>/user/test/test.sql</script>
            <file>hive-config.xml#hive-config.xml</file>
        </hive>
        <ok to="end"/>
        <error to="kill"/>
    </action>
    <kill name="kill">
        <message>Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
    </kill>
    <end name="end"/>
</workflow-app>


${jobTracker}
${nameNode}
/user/test/hive-config.xml
/user/test/test.sql
hive-config.xml#hive-config.xml
操作失败，错误消息[${wf:errorMessage（wf:lastErrorNode（））}]

此问题与nodeManager有关。尝试在mapred site.xml文件中设置以下属性可修复此问题：

<property> <name>mapred.tasktracker.map.tasks.maximum</name> <value>50 </value> </property> <property> <name>mapred.tasktracker.map.tasks.maximum</name> <value>50 </value> </property>

mapred.tasktracker.map.tasks.max 50 mapred.tasktracker.map.tasks.max 50

这与此问题类似：
请共享与此配置单元操作相关的oozie工作流我将指定内容添加到“/etc/hadoop/conf.cloudera.thread/mapred site.xml”并重新启动了Thread服务，但解决方案对我不起作用。