Spring boot 集成-ApacheFlink+;弹簧靴

Spring boot 集成-ApacheFlink+;弹簧靴,spring-boot,apache-flink,Spring Boot,Apache Flink,我正在测试ApacheFlink和SpringBoot之间的集成,在IDE上运行它们是可以的,但是当我尝试在ApacheFlink集群上运行时,我遇到了一个与ClassLoader相关的异常 课程非常简单: BootFlinkApplication @SpringBootApplication @ComponentScan("com.example.demo") public class BootFlinkApplication { public static void main(St

我正在测试ApacheFlink和SpringBoot之间的集成,在IDE上运行它们是可以的,但是当我尝试在ApacheFlink集群上运行时,我遇到了一个与ClassLoader相关的异常

课程非常简单:

BootFlinkApplication

@SpringBootApplication
@ComponentScan("com.example.demo")
public class BootFlinkApplication {

    public static void main(String[] args) {
        System.out.println("some test");
        SpringApplication.run(BootFlinkApplication.class, args);
    }
}
FlinkTest

@Service
public class FlinkTest {
    @PostConstruct
    public void init() {
        StreamExecutionEnvironment see = StreamExecutionEnvironment.getExecutionEnvironment();

        see.fromElements(1, 2, 3, 4)
            .filter(new RemoveNumber3Filter()).print();

        try {
            see.execute();
        } catch (Exception e) {
            System.out.println("Error executing flink job: " + e.getMessage());
        }
    }
}
移除数字3过滤器

public class RemoveNumber3Filter implements FilterFunction<Integer> {

    @Override
    public boolean filter(Integer i) throws Exception {
        return i != 3;
    }

}
公共类RemoveNumber3Filter实现FilterFunction{
@凌驾
公共布尔筛选器(整数i)引发异常{
返回i!=3;
}
}
例外情况:

Caused by: org.apache.flink.runtime.client.JobExecutionException: Job execution failed.
        at org.apache.flink.runtime.jobmanager.JobManager$$anonfun$handleMessage$1$$anonfun$applyOrElse$6.apply$mcV$sp(JobManager.scala:897)
        at org.apache.flink.runtime.jobmanager.JobManager$$anonfun$handleMessage$1$$anonfun$applyOrElse$6.apply(JobManager.scala:840)
        at org.apache.flink.runtime.jobmanager.JobManager$$anonfun$handleMessage$1$$anonfun$applyOrElse$6.apply(JobManager.scala:840)
        at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
        at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
        at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
        at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)
        at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
        at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
        at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
        at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
    Caused by: org.apache.flink.streaming.runtime.tasks.StreamTaskException: Cannot load user class: com.example.demo.RemoveNumber3Filter
    ClassLoader info: URL ClassLoader:
        file: '/tmp/blobStore-850f3189-807e-4f8d-a8a6-3bd3c1bd76b4/job_eb93b239080b4d4e09f10f1e3605744d/blob_p-5fd56f3348976c0d333d680fde4a79573c21cd40-48ac0995eee11f38ce3ff4f890102af8' (valid JAR)
    Class not resolvable through given classloader.
        at org.apache.flink.streaming.api.graph.StreamConfig.getStreamOperator(StreamConfig.java:232)
        at org.apache.flink.streaming.runtime.tasks.OperatorChain.createChainedOperator(OperatorChain.java:355)
        at org.apache.flink.streaming.runtime.tasks.OperatorChain.createOutputCollector(OperatorChain.java:282)
        at org.apache.flink.streaming.runtime.tasks.OperatorChain.<init>(OperatorChain.java:126)
        at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:231)
        at org.apache.flink.runtime.taskmanager.Task.run(Task.java:718)
        at java.lang.Thread.run(Thread.java:748)
原因:org.apache.flink.runtime.client.JobExecutionException:作业执行失败。
在org.apache.flink.runtime.jobmanager.jobmanager$$anonfun$handleMessage$1$$anonfun$applyorlse$6.apply$mcV$sp(jobmanager.scala:897)
在org.apache.flink.runtime.jobmanager.jobmanager$$anonfun$handleMessage$1$$anonfun$applyorlse$6.apply(jobmanager.scala:840)
在org.apache.flink.runtime.jobmanager.jobmanager$$anonfun$handleMessage$1$$anonfun$applyorlse$6.apply(jobmanager.scala:840)
在scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
在scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
运行(AbstractDispatcher.scala:39)
在akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)
位于scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
位于scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
位于scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
在scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)中
原因:org.apache.flink.streaming.runtime.tasks.StreamTaskException:无法加载用户类:com.example.demo.RemoveNumber3Filter
类加载器信息:URL类加载器:
文件:'/tmp/blobStore-850f3189-807e-4f8d-a8a6-3bd3c1bd76b4/job_eb93b239080b4d4e09f10f1e3605744d/blob_p-5FD56F3348976C0D333D680FDE4A79573CD40-48ac0995eee11f38ce3ff4f890102af8'(有效JAR)
类无法通过给定的类加载器解析。
位于org.apache.flink.streaming.api.graph.StreamConfig.getStreamOperator(StreamConfig.java:232)
位于org.apache.flink.streaming.runtime.tasks.OperatorChain.createChainedOperator(OperatorChain.java:355)
位于org.apache.flink.streaming.runtime.tasks.OperatorChain.createOutputCollector(OperatorChain.java:282)
位于org.apache.flink.streaming.runtime.tasks.OperatorChain.(OperatorChain.java:126)
位于org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:231)
位于org.apache.flink.runtime.taskmanager.Task.run(Task.java:718)
运行(Thread.java:748)

您可能使用Spring Boot Maven插件()重新打包Jar以生成可执行Jar,但是,它使用了Apache Flink内部类加载器不支持的自定义引导布局。在您尝试部署的文件(.jar.original)旁边应该有一个原始jar文件,您可以使用它在Flink集群上进行部署


作为替代,您可以使用不同的方法生成包含所有依赖项的Jar,例如maven shade()

您看到这个页面了吗?是的,我看到了,但没什么帮助…你能检查一下
RemoveNumber3Filter
是包含在jar
blob_p-5fd56f3348976c033d680fde4a79573c21cd40-48ac0995eee11f38ce3ff4f89102af8中还是包含在你用来启动应用程序的jar中。是的。。。问题是SpringBoot创建的jar在boot-INF/classes文件夹中有类