Hive 容器启动exitCode=2的异常，可能与Flink SQL配置单元连接器有关 Flink（Scala）exitCode=2在mvn清洁包之后

我有一个简单的Flink作业，它从一个配置单元表的两列中读取

mysource

，将这些列相加，然后将结果写入另一个配置单元表

mysink

，其中

mysource

有两列

a bigint

和

b bigint

，而

mysink

只有一列

c bigint

作业提交成功，但我观察到它一直在重试

我点击每一次尝试，他们只是显示这一点

AM Container for appattempt_1607399514900_2511_001267 exited with exitCode: 2
For more detailed output, check application tracking page:http://cn-hz-h-test-data-flink00:8088/cluster/app/application_1607399514900_2511Then, click on links to logs of each attempt.
Diagnostics: Exception from container-launch.
Container id: container_e13_1607399514900_2511_1267_000001
Exit code: 2
Stack trace: ExitCodeException exitCode=2:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:582)
at org.apache.hadoop.util.Shell.run(Shell.java:479)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:773)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:212)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Container exited with a non-zero exit code 2
Failing this attempt

然而，“日志”没有有用的信息——它抱怨日志库，但我相信它们实际上是警告，而不是错误

LogType:jobmanager.err
Log Upload Time:Wed Apr 07 10:30:52 +0800 2021
LogLength:1010
Log Contents:
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/data/apache/hadoop/hadoop-2.7.3/logs/tmp/nm-local-dir/usercache/zhongtai/appcache/application_1607399514900_2509/filecache/10/featurepipelines-0.1.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]/
SLF4J: Found binding in [jar:file:/data/apache/hadoop/hadoop-2.7.3/logs/tmp/nm-local-dir/filecache/302/log4j-slf4j-impl-2.12.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/data/apache/hadoop/hadoop-2.7.3/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
log4j:WARN No appenders could be found for logger (org.apache.flink.runtime.entrypoint.ClusterEntrypoint).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
End of LogType:jobmanager.err

LogType:jobmanager.out
Log Upload Time:Wed Apr 07 10:30:52 +0800 2021
LogLength:0
Log Contents:
End of LogType:jobmanager.out

这是用Scala编写的作业

import org.apache.flink.streaming.api.scala.streamexecutionenEnvironment
导入org.apache.flink.table.api.{EnvironmentSettings，sqldial}
导入org.apache.flink.table.api.bridge.scala.StreamTableEnvironment
导入org.apache.flink.table.catalog.hive.HiveCatalog
对象HiveToyExample{
def main（参数：数组[字符串]）：单位={
val settings=EnvironmentSettings.newInstance.build
val execEnv=StreamExecutionEnvironment.getExecutionEnvironment
val tableEnv=StreamTableEnvironment.create（execEnv，设置）
val hiveCatalog=新的hiveCatalog(
“我的蜂巢”，
“爱因特拉”，
“/data/apache/hive/apache-hive-2.1.0-bin/conf/”
)
tableEnv.registerCatalog（“myhive”，hiveCatalog）
tableEnv.useCatalog（“myhive”）
tableEnv.getConfig.setsqldialent（sqldialent.DEFAULT）
tableEnv
.executeSql（“”）
|插入mysink
|选择a+b
|来自mysource
|“.stripMargin）
}
}

这是pom.xml文件

4.0.0
例
，这可以通过查看我的pom进行验证

如果您正在构建自己的程序，则mvn文件中需要以下依赖项。建议不要在生成的jar文件中包含这些依赖项。您应该在运行时添加上述依赖项


org.apache.flink
:

添加依赖项的推荐方法是使用绑定的jar。只有当捆绑的罐子不能满足您的需要时，才应使用单独的罐子

我错过了什么

mvn clean package

flink run \
--yarnname scalaflink-hive-test \
-m yarn-cluster \
-yarnqueue datadev \
--class featurepipelines.ingestion.HiveToyExample \
./featurepipelines-0.1.1.jar

flink run \
--yarnname pyflink-hive-test \
-m yarn-cluster \
-yD yarn.application.queue=tech_platform \
-pyarch pyflink1.12.0.zip \
-pyexec /data/software/pyflink1.12.0/bin/python \
-py /data/home/pal-flink/chenyisheng14418/feature-pipelines/pyflink/hive.py

<!-- Flink Dependency -->
<dependency>
  <groupId>org.apache.flink</groupId>
  <artifactId>flink-connector-hive_2.11</artifactId>
  <version>1.12.0</version>
  <scope>provided</scope>
</dependency>

<dependency>
  <groupId>org.apache.flink</groupId>
  <artifactId>flink-table-api-java-bridge_2.11</artifactId>
  <version>1.12.0</version>
  <scope>provided</scope>
</dependency>

<!-- Hive Dependency -->
<dependency>
    <groupId>org.apache.hive</groupId>
    <artifactId>hive-exec</artifactId>
    <version>${hive.version}</version>
    <scope>provided</scope>
</dependency>