Warning: file_get_contents(/data/phpspider/zhask/data//catemap/9/java/377.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
如何使用Java代码向Flink集群提交作业?_Java_Deployment_Apache Flink_Flink Streaming - Fatal编程技术网

如何使用Java代码向Flink集群提交作业?

如何使用Java代码向Flink集群提交作业?,java,deployment,apache-flink,flink-streaming,Java,Deployment,Apache Flink,Flink Streaming,我已经将一个包含应用程序代码的胖jar上传到Flink集群中所有节点的/lib文件夹中。我试图从一个单独的java应用程序开始Flink工作,但找不到一个好方法 我目前发现的最接近解决方案的是监控RESTAPI,它有一个。但是,这只允许您运行通过作业上载功能提交的作业 我在flink客户端模块中看到了ClusterClient.java,但没有看到任何关于如何使用它的示例 任何关于某人如何通过java代码成功提交作业的示例都将不胜感激 您可以使用RestClusterClient运行指向Flin

我已经将一个包含应用程序代码的胖jar上传到Flink集群中所有节点的/lib文件夹中。我试图从一个单独的java应用程序开始Flink工作,但找不到一个好方法

我目前发现的最接近解决方案的是监控RESTAPI,它有一个。但是,这只允许您运行通过作业上载功能提交的作业

我在flink客户端模块中看到了ClusterClient.java,但没有看到任何关于如何使用它的示例


任何关于某人如何通过java代码成功提交作业的示例都将不胜感激

您可以使用
RestClusterClient
运行指向Flink作业的
PackagedProgram
。如果您的工作接受某些参数,您可以传递它们

以下是在
localhost:8081
上运行的独立群集的示例:

// import org.apache.flink.api.common.JobSubmissionResult;
// import org.apache.flink.client.deployment.StandaloneClusterId;
// import org.apache.flink.client.program.PackagedProgram;
// import org.apache.flink.client.program.rest.RestClusterClient;
// import org.apache.flink.configuration.Configuration;
// import org.apache.flink.configuration.JobManagerOptions;
// import org.apache.flink.configuration.RestOptions;

String clusterHost = "localhost";
int clusterPort = 8081;

Configuration config = new Configuration();
config.setString(JobManagerOptions.ADDRESS, clusterHost);
config.setInteger(RestOptions.PORT, clusterPort);

String jarFilePath = "/opt/flink/examples/streaming/SocketWindowWordCount.jar";
String[] args = new String[]{ "--port", "9000" };
PackagedProgram packagedProgram = new PackagedProgram(new File(jarFilePath), args);

RestClusterClient<StandaloneClusterId> client =
         new RestClusterClient<StandaloneClusterId>(config, StandaloneClusterId.getInstance());

int parallelism = 1;
JobSubmissionResult result = client.run(packagedProgram,  parallelism);
//导入org.apache.flink.api.common.JobSubmissionResult;
//导入org.apache.flink.client.deployment.StandaloneClusterId;
//导入org.apache.flink.client.program.PackagedProgram;
//导入org.apache.flink.client.program.rest.RestClusterClient;
//导入org.apache.flink.configuration.configuration;
//导入org.apache.flink.configuration.JobManagerOptions;
//导入org.apache.flink.configuration.RestOptions;
字符串clusterHost=“localhost”;
int clusterPort=8081;
配置配置=新配置();
config.setString(JobManagerOptions.ADDRESS,clusterHost);
config.setInteger(RestOptions.PORT、clusterPort);
字符串jarFilePath=“/opt/flink/examples/streaming/SocketWindowWordCount.jar”;
字符串[]args=新字符串[]{--port”,“9000”};
PackagedProgram PackagedProgram=新的PackagedProgram(新文件(jarFilePath),args);
RestClusterClient客户端=
新的RestClusterClient(config,StandaloneClusterId.getInstance());
int并行度=1;
JobSubmissionResult=client.run(packagedProgram,parallelism);

这似乎适用于1.10版

private static final int PARALLELISM = 8;
private static final Configuration FLINK_CONFIG = new Configuration();

void foo() throws Exception {
    FLINK_CONFIG.setString(JobManagerOptions.ADDRESS, "localhost");
    FLINK_CONFIG.setInteger(RestOptions.PORT, 8081);
    FLINK_CONFIG.setInteger(RestOptions.RETRY_MAX_ATTEMPTS, 3);

    RestClusterClient<StandaloneClusterId> flinkClient = new RestClusterClient<>(FLINK_CONFIG, StandaloneClusterId.getInstance());

    String jar = "/path/to/jar";
    String[] args = new String[]{"..."};
    PackagedProgram program = PackagedProgram.newBuilder()
            .setJarFile(new File(jar))
            .setArguments(args)
            .build();

    JobGraph jobGraph = PackagedProgramUtils.createJobGraph(program, FLINK_CONFIG, PARALLELISM, false);

    JobID jobId = flinkClient.submitJob(jobGraph).get();
    ...
}
private static final int PARALLELISM=8;
私有静态最终配置FLINK_CONFIG=新配置();
void foo()引发异常{
FLINK_CONFIG.setString(JobManagerOptions.ADDRESS,“localhost”);
FLINK_CONFIG.setInteger(RestOptions.PORT,8081);
FLINK_CONFIG.setInteger(RestOptions.RETRY_MAX_尝试次数,3);
RestClusterClient flinkClient=新的RestClusterClient(FLINK_CONFIG,StandaloneClusterId.getInstance());
String jar=“/path/to/jar”;
字符串[]args=新字符串[]{“…”};
PackagedProgram程序=PackagedProgram.newBuilder()
.setJarFile(新文件(jar))
.setArguments(参数)
.build();
JobGraph JobGraph=PackagedProgramUtils.createJobGraph(程序,FLINK_配置,并行,false);
JobID JobID=flinkClient.submitJob(jobGraph.get();
...
}

您的解决方案适用于flink 1.9,但不适用于1.10,因为我找不到
client.run()
方法。你也可以提供如何在1.10也做?谢谢,有没有任何方式提交给纱线?