Hadoop 时间轴服务v2无法启动

我在AWS上有一个测试HDP集群设置，用于评估项目。Ambari UI报告了一些错误，当我检查这些错误并在必要时重新启动服务时，我遇到了一些问题。启动Thread的Timeline Service Reader V2时，我得到一个错误

2018-08-10 15:51:06,400 INFO  [main] client.RpcRetryingCallerImpl: Call exception, tries=15, retries=15, started=129034 ms ago, cancelled=false, msg=Call to HOSTNAME/IPADDRESS:17020 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: HOSTNAME/IPADDRESS:17020, details=row 'prod.timelineservice.entity' on table 'hbase:meta' at region=hbase:meta,,1.1588230740, hostname=HOSTNAME,17020,1533827052949, seqNum=-1

最终导致

stderr: 
Traceback (most recent call last):
  File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 982, in restart
    self.status(env)
  File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/YARN/package/scripts/timelinereader.py", line 88, in status
    check_process_status(pid_file)
  File "/usr/lib/ambari-agent/lib/resource_management/libraries/functions/check_process_status.py", line 43, in check_process_status
    raise ComponentIsNotRunning()
ComponentIsNotRunning

The above exception was the cause of the following exception:

Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/YARN/package/scripts/timelinereader.py", line 108, in <module>
    ApplicationTimelineReader().execute()
  File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 353, in execute
    method(env)
  File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 993, in restart
    self.start(env, upgrade_type=upgrade_type)
  File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/YARN/package/scripts/timelinereader.py", line 51, in start
    hbase(action='start')
  File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/YARN/package/scripts/hbase_service.py", line 80, in hbase
    createTables()
  File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/YARN/package/scripts/hbase_service.py", line 147, in createTables
    logoutput=True)
  File "/usr/lib/ambari-agent/lib/resource_management/core/base.py", line 166, in __init__
    self.env.run()
  File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py", line 160, in run
    self.run_action(resource, action)
  File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py", line 124, in run_action
    provider_action()
  File "/usr/lib/ambari-agent/lib/resource_management/core/providers/system.py", line 263, in action_run
    returns=self.resource.returns)
  File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 72, in inner
    result = function(command, **kwargs)
  File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 102, in checked_call
    tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy, returns=returns)
  File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 150, in _call_wrapper
    result = _call(command, **kwargs_copy)
  File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 308, in _call
    raise ExecuteTimeoutException(err_msg)
resource_management.core.exceptions.ExecuteTimeoutException: Execution of 'ambari-sudo.sh su yarn-ats -l -s /bin/bash -c 'export  PATH='"'"'/usr/sbin:/sbin:/usr/lib/ambari-server/*:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/var/lib/ambari-agent'"'"' ; sleep 10;export HBASE_CLASSPATH_PREFIX=/usr/hdp/3.0.0.0-1634/hadoop-yarn/timelineservice/*; /usr/hdp/3.0.0.0-1634/hbase/bin/hbase --config /usr/hdp/3.0.0.0-1634/hadoop/conf/embedded-yarn-ats-hbase org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator -Dhbase.client.retries.number=35 -create -s'' was killed due timeout after 300 seconds

stderr:
回溯（最近一次呼叫最后一次）：
文件“/usr/lib/ambari-agent/lib/resource_-management/libraries/script/script.py”，第982行，重新启动
自我状态（env）
文件“/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/warn/package/scripts/timelinereader.py”，第88行，处于状态
检查进程状态（pid文件）
文件“/usr/lib/ambari agent/lib/resource\u management/libraries/functions/check\u process\u status.py”，第43行，处于check\u process\u状态
提升部件耳轴（）
组件耳轴
上述异常是以下异常的原因：
回溯（最近一次呼叫最后一次）：
文件“/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/warn/package/scripts/timelinereader.py”，第108行，在
ApplicationTimelineReader（）.execute（）
文件“/usr/lib/ambari agent/lib/resource_management/libraries/script/script.py”，第353行，在execute中
方法（env）
文件“/usr/lib/ambari-agent/lib/resource_-management/libraries/script/script.py”，第993行，重新启动
self.start（环境，升级类型=升级类型）
文件“/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/warn/package/scripts/timelinereader.py”，第51行，开头
hbase（action='start'）
hbase中的文件“/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/thread/package/scripts/hbase_-service.py”，第80行
createTables（）
createTables中的文件“/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/thread/package/scripts/hbase_-service.py”，第147行
logoutput=True）
文件“/usr/lib/ambari-agent/lib/resource\u-management/core/base.py”，第166行，在__
self.env.run（）文件
文件“/usr/lib/ambari-agent/lib/resource_-management/core/environment.py”，第160行，正在运行
self.run_操作（资源、操作）
文件“/usr/lib/ambari-agent/lib/resource\u-management/core/environment.py”，第124行，在运行操作中
提供者_操作（）
文件“/usr/lib/ambari-agent/lib/resource\u-management/core/providers/system.py”，第263行，运行中
returns=self.resource.returns）
文件“/usr/lib/ambari-agent/lib/resource_-management/core/shell.py”，第72行，内部
结果=功能（命令，**kwargs）
文件“/usr/lib/ambari-agent/lib/resource\u-management/core/shell.py”，第102行，在checked\u调用中
trys=trys，try\u sleep=try\u sleep，timeout\u kill\u strategy=timeout\u kill\u strategy，returns=returns）
文件“/usr/lib/ambari-agent/lib/resource\u-management/core/shell.py”，第150行，在调用包装中
结果=_调用（命令，**kwargs_复制）
文件“/usr/lib/ambari-agent/lib/resource\u-management/core/shell.py”，第308行，in\u-call
引发ExecuteTimeoutException（错误消息）
resource_management.core.exceptions.ExecuteTimeoutException:execute'ambari-sudo.sh su-ats-l-s/bin/bash-c'导出路径='''''''''''''/usr/sbin:/sbin:/usr/lib/ambari-server/*：/usr/local sbin:/usr/local/bin:/usr/sbin:/usr/bin:/var/lib/ambari-agent'''；睡眠10；导出HBASE_CLASSPATH_PREFIX=/usr/hdp/3.0.0.0-1634/hadoop Thread/timelineservice/*/usr/hdp/3.0.0.0-1634/hbase/bin/hbase--config/usr/hdp/3.0.0.0-1634/hadoop/conf/embedded-thread-ats-hbase org.apache.hadoop.thread.server.timelineservice.storage.TimelineSchemaCreator-Dhbase.client.retries.number=35-create-s“”在300秒后由于超时而被终止

---

哪个组件需要重新启动才能使纱线恢复正常状态，以及将来调试问题的正确方法是什么？

如果进入“后台操作”（Ambari UI中的齿轮图标），则转到Timeline Service V2启动链接（您可能需要首先单击运行Timeline服务的计算机才能到达那里），您应该在右上角有“复制”和“打开”的链接。这些链接有望向您显示更详细的错误日志

---

在我的例子中，Timeline Service V2无法启动，因为系统上没有足够的内存。它是一个小型VM群集，仅用于在每台机器上使用2GB RAM。我通过更详细的错误日志发现，它出现了内存不足错误，因此当我将VM内存增加到4GB时，它能够运行。我的best猜测您在运行Ambari UI的主NameNode上的内存不足。似乎需要4GB+左右的内存，具体取决于您在主NameNode上运行的服务数量。

这可能是防火墙问题。关闭防火墙或释放您尝试运行作业的端口。因为连接失败ed问题直接指出了这个方向。更多信息：感谢@Abhinav这是我最初的想法，但防火墙规则在管理器和代理节点之间是开放的。还有什么可能抛出“拒绝连接”呢该组件出错？另一个可能的原因可能是系统中的主机条目。如果服务器无法解析主机名，那么它也可能发送“连接被拒绝”错误。@haiqus我也面临同样的错误。你找到解决方案了吗？有人能解决这个问题吗？