Java 将历史事件发布到ATSv2时出错

我们正在尝试将历史事件发布到。我们已启用Thread.timeline-service.2.0f版

我们已经成功地将jira中提到的补丁应用到tez-0.9.2源代码中

if (conf.getBoolean(YarnConfiguration.TIMELINE_SERVICE_ENABLED,
    YarnConfiguration.DEFAULT_TIMELINE_SERVICE_ENABLED)) {

  timelineV2Client =
  TimelineV2Client.createTimelineClient(appContext.getApplicationID());
  timelineV2Client.init(conf);

  // assumption is at this point AMRMClient is created! So callback handler is set to null
  TezAMRMClientAsyncProvider.createAMRMClientAsync(1000, null)
      .registerTimelineV2Client(timelineV2Client);

  LOG.info("timelineV2Client inited.");
} else {
  this.timelineV2Client = null;
  if (conf.get(TezConfiguration.TEZ_HISTORY_LOGGING_SERVICE_CLASS, "")
      .equals(ATSV2HistoryLoggingService.class.getName())) {
    LOG.warn(ATSV2HistoryLoggingService.class.getName()
        + " is disabled due to Timeline Service being disabled, "
        + YarnConfiguration.TIMELINE_SERVICE_ENABLED + " set to false");
 }

当我们使用下面的命令运行tez作业时

    hadoop jar /usr/lib/tez/tez-tests-0.9.2.jar mrrsleep -m 10 -r 20

我们得到空指针异常以及dag持续等待，如下所示

ag.app.web.WebUIService] |http.HttpServer2|: Jetty bound to port 34923
 [INFO] [ServiceThread:org.apache.tez.dag.app.web.WebUIService]|mortbay.log|: jetty-6.1.26
[INFO] [ServiceThread:org.apache.tez.dag.app.web.WebUIService]|mortbay.log|: Extract jar:file:/usr/lib/hadoop/hadoop-common-2.10.1-tests.jar!/webapps/ to /hadoop/yarn/nm-local-dir/usercache/yadavsanjeev/appcache/application_1620837070665_0001/container_e01_1620837070665_0001_01_000001/tmp/Jetty_0_0_0_0_34923_webapps____fsve58/webapp
 [INFO] [ServiceThread:org.apache.tez.dag.app.web.WebUIService] |mortbay.log|: Started HttpServer2$SelectChannelConnectorWithSafeStartup@0.0.0.0:34923
 [INFO] [ServiceThread:org.apache.tez.dag.app.web.WebUIService] |webapp.WebApps|: Web app  started at 34923
 [INFO] [ServiceThread:org.apache.tez.dag.app.web.WebUIService] |web.WebUIService|: Instantiated WebUIService at http://mysandbox.internal:34923/ui/
[INFO] [ServiceThread:org.apache.tez.dag.app.rm.TaskSchedulerManager] |rm.TaskSchedulerManager|: Creating YARN TaskScheduler: org.apache.tez.dag.app.rm.YarnTaskSchedulerService
 [INFO] [ServiceThread:org.apache.tez.dag.app.rm.TaskSchedulerManager] |conf.Configuration|: resource-types.xml not found
 [INFO] [ServiceThread:org.apache.tez.dag.app.rm.TaskSchedulerManager] |resource.ResourceUtils|: Unable to find 'resource-types.xml'.
 [INFO] [ServiceThread:org.apache.tez.dag.app.rm.TaskSchedulerManager] |resource.ResourceUtils|: Adding resource type - name = memory-mb, units = Mi, type = COUNTABLE
[INFO] [ServiceThread:org.apache.tez.dag.app.rm.TaskSchedulerManager] |resource.ResourceUtils|: Adding resource type - name = vcores, units = , type = COUNTABLE
 [INFO] [ServiceThread:org.apache.tez.dag.app.rm.TaskSchedulerManager] |Configuration.deprecation|: io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum
 [INFO] [ServiceThread:org.apache.tez.dag.app.rm.TaskSchedulerManager] |rm.YarnTaskSchedulerService|: YarnTaskScheduler initialized with configuration: maxRMHeartbeatInterval: 1000, containerReuseEnabled: true, reuseRackLocal: true, reuseNonLocal: false, localitySchedulingDelay: 250, preemptionPercentage: 10, preemptionMaxWaitTime: 60000, numHeartbeatsBetweenPreemptions: 3, idleContainerMinTimeout: 5000, idleContainerMaxTimeout: 10000, sessionMinHeldContainers: 0
 [INFO] [ServiceThread:org.apache.tez.dag.app.rm.TaskSchedulerManager] |client.RMProxy|: Connecting to ResourceManager at mysandbox/10.0.0.79:8030
 [INFO] [main] |HistoryEventHandler.criticalEvents|: [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1620837070665_0001_000001, startTime=1620837589967
 [INFO] [main] |app.DAGAppMaster|: In Non-Session mode.
 [INFO] [ATSV2HistoryEventHandlingThread] |ats.ATSV2HistoryLoggingService|: Publishing entity TimelineEntity[type='TEZ_APPLICATION_ATTEMPT', id='tez_appattempt_1620837070665_0001_000001']
 [INFO] [main] |app.DAGAppMaster|: Merging AM credentials into DAG credentials
 [ERROR] [AMRM Callback Handler Thread] |yarn.YarnUncaughtExceptionHandler|: Thread Thread[AMRM Callback Handler Thread,5,main] threw an Exception.
java.lang.NullPointerException
at org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl$CallbackHandlerThread.run(AMRMClientAsyncImpl.java:371)
 [INFO] [main] |app.DAGAppMaster|: Generating DAG graphviz file, dagId=dag_1620837070665_0001_1, filePath=/var/log/hadoop-yarn/userlogs/application_1620837070665_0001/container_e01_1620837070665_0001_01_000001/dag_1620837070665_0001_1.dot
 [INFO] [main] |common.TezUtilsInternal|: Redirecting log file based on addend: dag_1620837070665_0001_1

正如jira中提到的，假设此时创建了AMRMClient，所以回调处理程序设置为null，但不清楚这里如何使用它

因此，timeline service addres为空，无法发布事件