Java 将作业部署到Flink群集
我想通过java应用程序将配置好的flink作业部署到独立集群。我想用RestClusterClient来完成它 据我所知,我可以通过以下方式完成:Java 将作业部署到Flink群集,java,apache-flink,Java,Apache Flink,我想通过java应用程序将配置好的flink作业部署到独立集群。我想用RestClusterClient来完成它 据我所知,我可以通过以下方式完成: 用我的作业创建一个jar(例如,通过数据转换从一个卡夫卡主题到另一个卡夫卡主题的简单流) 通过使用PackagedProgramUtils从jar文件创建JobGraph 初始化RestClusterClient并将JobGraph提交到集群 PackagedProgram PackagedProgram=PackagedProgram.newBu
PackagedProgram PackagedProgram=PackagedProgram.newBuilder()
.setJarFile(新文件(“path/to/my/jar/File”))
.setArguments(参数)
.build();
JobGraph JobGraph=PackagedProgramUtils.createJobGraph(packagedProgram,flinkConfiguration,1,false);
try(RestClusterClient=new RestClusterClient(flinkConfiguration,StandaloneClusterId.getInstance()){
client.submitJob(jobGraph.get();
}
但是,我想配置我的作业并将其部署到一个java应用程序中的集群中。我找到了以下方法:
StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
// add source, any data processing and sink
StreamGraph streamGraph = graphFlow.getConfiguredEnvironment().getStreamGraph("myGraphName", false);
JobGraph graph = streamGraph.getJobGraph();
try (RestClusterClient<StandaloneClusterId> client = new RestClusterClient<>(flinkConfiguration, StandaloneClusterId.getInstance())) {
client.submitJob(jobGraph).get();
}
StreamExecutionEnvironment env=StreamExecutionEnvironment.getExecutionEnvironment();
//添加源、任何数据处理和接收器
StreamGraph StreamGraph=graphFlow.GetConfigureDenEnvironment().getStreamGraph(“myGraphName”,false);
JobGraph=streamGraph.getJobGraph();
try(RestClusterClient=new RestClusterClient(flinkConfiguration,StandaloneClusterId.getInstance()){
client.submitJob(jobGraph.get();
}
此解决方案似乎不可接受,因为StreamExecutionEnvironment方法被注释为@Internal。还有哪些方法可用于构建可通过RestClusterClient上传的作业图对象
由于我需要将作业上载到flink cluster,因此无法使用RemoteStreamEnvironment
Flink版本是1.11.2
StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
// add source, any data processing and sink
StreamGraph streamGraph = graphFlow.getConfiguredEnvironment().getStreamGraph("myGraphName", false);
JobGraph graph = streamGraph.getJobGraph();
try (RestClusterClient<StandaloneClusterId> client = new RestClusterClient<>(flinkConfiguration, StandaloneClusterId.getInstance())) {
client.submitJob(jobGraph).get();
}