Warning: file_get_contents(/data/phpspider/zhask/data//catemap/2/tensorflow/5.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Java 将作业部署到Flink群集_Java_Apache Flink - Fatal编程技术网

Java 将作业部署到Flink群集

Java 将作业部署到Flink群集,java,apache-flink,Java,Apache Flink,我想通过java应用程序将配置好的flink作业部署到独立集群。我想用RestClusterClient来完成它 据我所知,我可以通过以下方式完成: 用我的作业创建一个jar(例如,通过数据转换从一个卡夫卡主题到另一个卡夫卡主题的简单流) 通过使用PackagedProgramUtils从jar文件创建JobGraph 初始化RestClusterClient并将JobGraph提交到集群 PackagedProgram PackagedProgram=PackagedProgram.newBu

我想通过java应用程序将配置好的flink作业部署到独立集群。我想用RestClusterClient来完成它

据我所知,我可以通过以下方式完成:

  • 用我的作业创建一个jar(例如,通过数据转换从一个卡夫卡主题到另一个卡夫卡主题的简单流)
  • 通过使用PackagedProgramUtils从jar文件创建JobGraph
  • 初始化RestClusterClient并将JobGraph提交到集群
  • PackagedProgram PackagedProgram=PackagedProgram.newBuilder()
    .setJarFile(新文件(“path/to/my/jar/File”))
    .setArguments(参数)
    .build();
    JobGraph JobGraph=PackagedProgramUtils.createJobGraph(packagedProgram,flinkConfiguration,1,false);
    try(RestClusterClient=new RestClusterClient(flinkConfiguration,StandaloneClusterId.getInstance()){
    client.submitJob(jobGraph.get();
    }
    
    但是,我想配置我的作业并将其部署到一个java应用程序中的集群中。我找到了以下方法:

    StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
    // add source, any data processing and sink
    StreamGraph streamGraph = graphFlow.getConfiguredEnvironment().getStreamGraph("myGraphName", false);
    JobGraph graph = streamGraph.getJobGraph();
    try (RestClusterClient<StandaloneClusterId> client = new RestClusterClient<>(flinkConfiguration, StandaloneClusterId.getInstance())) {
        client.submitJob(jobGraph).get();
    }
    
    StreamExecutionEnvironment env=StreamExecutionEnvironment.getExecutionEnvironment();
    //添加源、任何数据处理和接收器
    StreamGraph StreamGraph=graphFlow.GetConfigureDenEnvironment().getStreamGraph(“myGraphName”,false);
    JobGraph=streamGraph.getJobGraph();
    try(RestClusterClient=new RestClusterClient(flinkConfiguration,StandaloneClusterId.getInstance()){
    client.submitJob(jobGraph.get();
    }
    
    此解决方案似乎不可接受,因为StreamExecutionEnvironment方法被注释为@Internal。还有哪些方法可用于构建可通过RestClusterClient上传的作业图对象

    由于我需要将作业上载到flink cluster,因此无法使用RemoteStreamEnvironment

    Flink版本是1.11.2

    StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
    // add source, any data processing and sink
    StreamGraph streamGraph = graphFlow.getConfiguredEnvironment().getStreamGraph("myGraphName", false);
    JobGraph graph = streamGraph.getJobGraph();
    try (RestClusterClient<StandaloneClusterId> client = new RestClusterClient<>(flinkConfiguration, StandaloneClusterId.getInstance())) {
        client.submitJob(jobGraph).get();
    }