Spring SCDF-在不同的模式中创建元数据表，并使用不同的模式处理批处理作业

我使用的是

springbatch

，它从

Oracle

加载数据并将其放入

MongoDB

。我希望使用Spring云数据流，但是

SCDF

不支持

MongoDB

如果我们可以将

SCDF

元数据维护到

Postgres

（因为它有很好的支持）或H2中，有什么办法吗？在本例中，我使用H2，并运行实际的批处理作业，将数据从Oracle加载到

MongoDB

当我尝试运行这个组合时，我得到了下面的错误，当然一些配置更改需要修改。有什么快速的解决办法吗

错误：

  org.springframework.batch.item.ItemStreamException: Failed to initialize the reader
    at org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader.open(AbstractItemCountingItemStreamItemReader.java:152) ~[spring-batch-infrastructure-4.1.1.RELEASE.jar!/:4.1.1.RELEASE]
    at org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader$$FastClassBySpringCGLIB$$ebb633d0.invoke(<generated>) ~[spring-batch-infrastructure-4.1.1.RELEASE.jar!/:4.1.1.RELEASE]
    at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) ~[spring-core-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:749) ~[spring-aop-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) [spring-aop-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
    at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:136) ~[spring-aop-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
    at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:124) ~[spring-aop-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) [spring-aop-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
    at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:688) ~[spring-aop-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
    at org.springframework.batch.item.database.JdbcCursorItemReader$$EnhancerBySpringCGLIB$$3c62d122.open(<generated>) ~[spring-batch-infrastructure-4.1.1.RELEASE.jar!/:4.1.1.RELEASE]
    at org.springframework.batch.item.support.CompositeItemStream.open(CompositeItemStream.java:103) ~[spring-batch-infrastructure-4.1.1.RELEASE.jar!/:4.1.1.RELEASE]
    at org.springframework.batch.core.step.tasklet.TaskletStep.open(TaskletStep.java:311) ~[spring-batch-core-4.1.1.RELEASE.jar!/:4.1.1.RELEASE]
    at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:200) ~[spring-batch-core-4.1.1.RELEASE.jar!/:4.1.1.RELEASE]
    at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:148) [spring-batch-core-4.1.1.RELEASE.jar!/:4.1.1.RELEASE]
    at org.springframework.batch.core.job.AbstractJob.handleStep(AbstractJob.java:399) [spring-batch-core-4.1.1.RELEASE.jar!/:4.1.1.RELEASE]
    at org.springframework.batch.core.job.SimpleJob.doExecute(SimpleJob.java:135) [spring-batch-core-4.1.1.RELEASE.jar!/:4.1.1.RELEASE]
    at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:313) [spring-batch-core-4.1.1.RELEASE.jar!/:4.1.1.RELEASE]
    at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:144) [spring-batch-core-4.1.1.RELEASE.jar!/:4.1.1.RELEASE]
    at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50) [spring-core-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
    at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:137) [spring-batch-core-4.1.1.RELEASE.jar!/:4.1.1.RELEASE]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_171]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_171]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_171]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_171]
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343) [spring-aop-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) [spring-aop-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) [spring-aop-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
    at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:127) [spring-batch-core-4.1.1.RELEASE.jar!/:4.1.1.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) [spring-aop-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) [spring-aop-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
    at com.sun.proxy.$Proxy66.run(Unknown Source) [na:na]
    at com.mastercard.customer.data.management.refdata.ReferenceMongoBatchApplication.run(ReferenceMongoBatchApplication.java:63) [classes!/:1.0]
    at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:813) [spring-boot-2.1.3.RELEASE.jar!/:2.1.3.RELEASE]
    at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:797) [spring-boot-2.1.3.RELEASE.jar!/:2.1.3.RELEASE]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:324) [spring-boot-2.1.3.RELEASE.jar!/:2.1.3.RELEASE]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1260) [spring-boot-2.1.3.RELEASE.jar!/:2.1.3.RELEASE]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1248) [spring-boot-2.1.3.RELEASE.jar!/:2.1.3.RELEASE]
    at com.mastercard.customer.data.management.refdata.ReferenceMongoBatchApplication.main(ReferenceMongoBatchApplication.java:51) [classes!/:1.0]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_171]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_171]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_171]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_171]
    at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48) [reference-mongo-batch-1.0.jar:1.0]
    at org.springframework.boot.loader.Launcher.launch(Launcher.java:87) [reference-mongo-batch-1.0.jar:1.0]
    at org.springframework.boot.loader.Launcher.launch(Launcher.java:50) [reference-mongo-batch-1.0.jar:1.0]
    at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:51) [reference-mongo-batch-1.0.jar:1.0]

配置

@Configuration
public class EmployeeJob {
    @Value( "${spring.chunk.size}")
    private String chunkSize;

    @Autowired
    private JobBuilderFactory jobBuilderFactory;

    @Autowired
    private StepBuilderFactory stepBuilderFactory;

    @Autowired
    private JdbcCursorItemReader<Employee> EmployeeReader;

    @Autowired
    public AsyncItemProcessor<Employee, Employee> asyncItemProcessor;

    @Autowired
    public AsyncItemWriter<Employee> asyncItemWriter;

    @Bean
    public EmployeeStepExecuListner EmployeeStepExecuListner() {
        return new EmployeeStepExecuListner();
    }

    @Bean("readEmployeeJob")
    @Primary
    public Job readEmployeeJob() {
        return jobBuilderFactory.get("readEmployeeJob")
                .incrementer(new RunIdIncrementer())
                .start(EmployeeStepOne())
                .build();
    }

    @SuppressWarnings({ "unchecked", "rawtypes" })
    @Bean
    public Step EmployeeStepOne() {
        return stepBuilderFactory.get("EmployeeStepOne")
                .<Employee, Employee>chunk(Integer.parseInt(chunkSize))
                .reader(EmployeeReader)
                .processor((ItemProcessor) asyncItemProcessor)
                .writer(asyncItemWriter)
                .build();
    }
}

@Configuration
public class EmployeeBatchConfig {

    @Autowired
    @Qualifier(value="oracleDS")
    private DataSource dataSource;

    @Bean(destroyMethod = "")
    @StepScope
    public JdbcCursorItemReader<Employee> EmployeeReader() throws Exception {
        JdbcCursorItemReader<Employee> reader = new JdbcCursorItemReader<>();
        reader.setDataSource(this.dataSource);
        reader.setSql(SELECT_Employee_SQL);

        reader.setRowMapper(new EmployeeRowMapper());
        reader.afterPropertiesSet();
        return reader;
    }

    @Bean
    public ItemProcessor<Employee, Employee> EmployeeProcessor() {
        return new EmployeeProcessor();
    }

    @Bean
    public AsyncItemProcessor<Employee, Employee> asyncItemProcessor() throws Exception{
        AsyncItemProcessor<Employee, Employee> asyncItemProcessor = new AsyncItemProcessor<>();
        asyncItemProcessor.setDelegate(EmployeeProcessor());
        asyncItemProcessor.setTaskExecutor(new SimpleAsyncTaskExecutor());
        asyncItemProcessor.afterPropertiesSet();
        return asyncItemProcessor;
    }

    @Bean
    public EmployeeWriter EmployeeWriter() {
        return new EmployeeWriter();
    }


    @Bean
    public AsyncItemWriter<Employee> asyncItemWriter() throws Exception{
        AsyncItemWriter<Employee> asyncItemWriter = new AsyncItemWriter<>();
        asyncItemWriter.setDelegate(EmployeeWriter());
        asyncItemWriter.afterPropertiesSet();
        return asyncItemWriter;  
    }
}

@配置
公营雇员职位{
@值（${spring.chunk.size}）
私有字符串块大小；
@自动连线
私人JobBuilderFactory JobBuilderFactory；
@自动连线
私人StepBuilderFactory StepBuilderFactory；
@自动连线
私有JdbcCursorItemReader EmployeeReader；
@自动连线
公共AsyncItemProcessor AsyncItemProcessor；
@自动连线
公共AsyncItemWriter AsyncItemWriter；
@豆子
public employeesteppexecutilistner employeesteppexecutilistner（）{
返回新的EmployeeStepExecutListner（）；
}
@Bean（“readEmployeeJob”）
@初级的
公共职务readEmployeeJob（）{
返回jobBuilderFactory.get（“readEmployeeJob”）
.incrementer（新的RunIdIncrementer（））
.start（EmployeeStepOne（））
.build（）；
}
@SuppressWarnings（{“unchecked”，“rawtypes”}）
@豆子
公共步骤雇员步骤一（）{
返回stepBuilderFactory.get（“EmployeeStepOne”）
.chunk（整数.parseInt（chunkSize））
.读卡器（员工负责人）
.processor（（ItemProcessor）asyncItemProcessor）
.writer（asyncItemWriter）
.build（）；
}
}

配置

@Configuration
public class EmployeeJob {
    @Value( "${spring.chunk.size}")
    private String chunkSize;

    @Autowired
    private JobBuilderFactory jobBuilderFactory;

    @Autowired
    private StepBuilderFactory stepBuilderFactory;

    @Autowired
    private JdbcCursorItemReader<Employee> EmployeeReader;

    @Autowired
    public AsyncItemProcessor<Employee, Employee> asyncItemProcessor;

    @Autowired
    public AsyncItemWriter<Employee> asyncItemWriter;

    @Bean
    public EmployeeStepExecuListner EmployeeStepExecuListner() {
        return new EmployeeStepExecuListner();
    }

    @Bean("readEmployeeJob")
    @Primary
    public Job readEmployeeJob() {
        return jobBuilderFactory.get("readEmployeeJob")
                .incrementer(new RunIdIncrementer())
                .start(EmployeeStepOne())
                .build();
    }

    @SuppressWarnings({ "unchecked", "rawtypes" })
    @Bean
    public Step EmployeeStepOne() {
        return stepBuilderFactory.get("EmployeeStepOne")
                .<Employee, Employee>chunk(Integer.parseInt(chunkSize))
                .reader(EmployeeReader)
                .processor((ItemProcessor) asyncItemProcessor)
                .writer(asyncItemWriter)
                .build();
    }
}

@Configuration
public class EmployeeBatchConfig {

    @Autowired
    @Qualifier(value="oracleDS")
    private DataSource dataSource;

    @Bean(destroyMethod = "")
    @StepScope
    public JdbcCursorItemReader<Employee> EmployeeReader() throws Exception {
        JdbcCursorItemReader<Employee> reader = new JdbcCursorItemReader<>();
        reader.setDataSource(this.dataSource);
        reader.setSql(SELECT_Employee_SQL);

        reader.setRowMapper(new EmployeeRowMapper());
        reader.afterPropertiesSet();
        return reader;
    }

    @Bean
    public ItemProcessor<Employee, Employee> EmployeeProcessor() {
        return new EmployeeProcessor();
    }

    @Bean
    public AsyncItemProcessor<Employee, Employee> asyncItemProcessor() throws Exception{
        AsyncItemProcessor<Employee, Employee> asyncItemProcessor = new AsyncItemProcessor<>();
        asyncItemProcessor.setDelegate(EmployeeProcessor());
        asyncItemProcessor.setTaskExecutor(new SimpleAsyncTaskExecutor());
        asyncItemProcessor.afterPropertiesSet();
        return asyncItemProcessor;
    }

    @Bean
    public EmployeeWriter EmployeeWriter() {
        return new EmployeeWriter();
    }


    @Bean
    public AsyncItemWriter<Employee> asyncItemWriter() throws Exception{
        AsyncItemWriter<Employee> asyncItemWriter = new AsyncItemWriter<>();
        asyncItemWriter.setDelegate(EmployeeWriter());
        asyncItemWriter.afterPropertiesSet();
        return asyncItemWriter;  
    }
}

@配置
公共类EmployeeBatchConfig{
@自动连线
@限定符（value=“oracleDS”）
私有数据源；
@Bean（方法=”）
@步进镜
公共JdbcCursorItemReader EmployeeReader（）引发异常{
JdbcCursorItemReader=新的JdbcCursorItemReader（）；
reader.setDataSource（this.dataSource）；
reader.setSql（选择_Employee_SQL）；
setRowMapper（newEmployeeRowMapper（））；
reader.afterPropertiesSet（）；
返回读取器；
}
@豆子
公共项目处理器EmployeeProcessor（）{
返回新的EmployeeProcessor（）；
}
@豆子
公共AsyncItemProcessor AsyncItemProcessor（）引发异常{
AsyncItemProcessor AsyncItemProcessor=新的AsyncItemProcessor（）；
setDelegate（EmployeeProcessor（））；
asyncItemProcessor.setTaskExecutor（新的SimpleAsyncTaskExecutor（））；
asyncItemProcessor.AfterPropertieSet（）；
返回异步项处理器；
}
@豆子
公共雇员撰稿人雇员撰稿人（）{
返回新的EmployeeWriter（）；
}
@豆子
公共AsyncItemWriter AsyncItemWriter（）引发异常{
AsyncItemWriter AsyncItemWriter=新建AsyncItemWriter（）；
asyncItemWriter.setDelegate（EmployeeWriter（））；
asyncItemWriter.AfterPropertieSet（）；
返回asyncItemWriter；
}
}

您的作业配置和数据源配置是什么样子的？@PhilipWrage-我已经添加了详细信息，如果需要其他详细信息，请告诉我