Spring batch 一步完成不同类型的弹簧批处理
我有一个从文件中读取记录的批处理作业。我想将上述记录转换为PojoA(所有字符串)。我希望运行每个记录并抛出一个验证器,以确保所有字段都存在。然后我想把PojoA变成PojoB。我遇到的问题是,我无法在中途更改对象的类型Spring batch 一步完成不同类型的弹簧批处理,spring-batch,Spring Batch,我有一个从文件中读取记录的批处理作业。我想将上述记录转换为PojoA(所有字符串)。我希望运行每个记录并抛出一个验证器,以确保所有字段都存在。然后我想把PojoA变成PojoB。我遇到的问题是,我无法在中途更改对象的类型 return getStepBuilder("downloadData") .<PojoA, PojoA>chunk(1000) .reader(pojoAReader()) .processor(pojoAValid
return getStepBuilder("downloadData")
.<PojoA, PojoA>chunk(1000)
.reader(pojoAReader())
.processor(pojoAValidator)
.writer(pojoAWriter)
.processor(pojoAToPojoBTransformer) <- issue here, <PojoA, PojoB>
.write(pojoBWriter)
.build();
返回getStepBuilder(“下载数据”)
.chunk(1000)
.reader(pojoAReader())
.处理器(pojoAValidator)
.作者(pojoAWriter)
.processor(pojoatopjobTransformer)不能声明两个处理器/写入程序,如:
.processor(pojoAValidator)
.writer(pojoAWriter)
.processor(pojoAToPojoBTransformer) <- issue here, <PojoA, PojoB>
.write(pojoBWriter)
处理器(pojoAValidator)
.作者(pojoAWriter)
.processor(pojoatopjobTransformer)整数),然后是processor2(整数->字符串):
导入java.util.array;
导入org.springframework.batch.core.Job;
导入org.springframework.batch.core.JobParameters;
导入org.springframework.batch.core.Step;
导入org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;
导入org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
导入org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
导入org.springframework.batch.core.launch.JobLauncher;
导入org.springframework.batch.item.ItemProcessor;
导入org.springframework.batch.item.ItemReader;
导入org.springframework.batch.item.ItemWriter;
导入org.springframework.batch.item.support.CompositeItemProcessor;
导入org.springframework.batch.item.support.ListItemReader;
导入org.springframework.beans.factory.annotation.Autowired;
导入org.springframework.context.ApplicationContext;
导入org.springframework.context.annotation.AnnotationConfigApplicationContext;
导入org.springframework.context.annotation.Bean;
导入org.springframework.context.annotation.Configuration;
@配置
@启用批处理
公共类MyJob{
@自动连线
私人建筑工地;
@自动连线
私人StepBuilderFactorySteps;
@豆子
public ItemReader ItemReader(){
返回新的ListItemReader(Arrays.asList(1,2,3,4,5,6,7,8,9,10));
}
@豆子
公共ItemWriter ItemWriter(){
退货项目->{
用于(字符串项:项){
System.out.println(“项=”+项);
}
};
}
@豆子
public ItemProcessor itemProcessor1(){
退货项目->项目+1;
}
@豆子
public ItemProcessor itemProcessor2(){
返回字符串::valueOf;
}
@豆子
public ItemProcessor compositeItemProcessor(){
CompositeItemProcessor CompositeItemProcessor=新的CompositeItemProcessor();
setDelegates(Arrays.asList(itemProcessor1(),itemProcessor2());
返回复合处理器;
}
@豆子
公共步骤(){
返回步骤。获取(“步骤”)
.chunk(5)
.reader(itemReader())
.processor(compositeItemProcessor())
.writer(itemWriter())
.build();
}
@豆子
公职{
返回作业。获取(“作业”)
.start(步骤())
.build();
}
公共静态void main(字符串[]args)引发异常{
ApplicationContext上下文=新注释ConfigApplicationContext(MyJob.class);
JobLauncher JobLauncher=context.getBean(JobLauncher.class);
Job=context.getBean(Job.class);
运行(作业,新作业参数());
}
}
我想将Integer对象写入DB表,验证它们,如果通过,则转换为String,然后将String对象写入DB表。我可以稍后再做,但是当我在建议的配置中添加一个整数编写器时,它抱怨我的IntegerWriter的格式不是String,这是正确的,它的类型是Integer。有可能绕过这个问题吗?在我上面的例子中,在复合处理器中添加writer(pojoAWriter)不会编译。考虑到这一点,我的用例确实违背了批处理的RPW范式;我基本上是在要求RWPW。我必须将其分解为两个步骤,并通过队列类型解决方案在这两个步骤之间传递数据。我很惊讶Batch还没有想出解决这个问题的方法。
import java.util.Arrays;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobParameters;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.batch.item.ItemProcessor;
import org.springframework.batch.item.ItemReader;
import org.springframework.batch.item.ItemWriter;
import org.springframework.batch.item.support.CompositeItemProcessor;
import org.springframework.batch.item.support.ListItemReader;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.ApplicationContext;
import org.springframework.context.annotation.AnnotationConfigApplicationContext;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
@Configuration
@EnableBatchProcessing
public class MyJob {
@Autowired
private JobBuilderFactory jobs;
@Autowired
private StepBuilderFactory steps;
@Bean
public ItemReader<Integer> itemReader() {
return new ListItemReader<>(Arrays.asList(1, 2, 3, 4, 5, 6, 7, 8, 9, 10));
}
@Bean
public ItemWriter<String> itemWriter() {
return items -> {
for (String item : items) {
System.out.println("item = " + item);
}
};
}
@Bean
public ItemProcessor<Integer, Integer> itemProcessor1() {
return item -> item + 1;
}
@Bean
public ItemProcessor<Integer, String> itemProcessor2() {
return String::valueOf;
}
@Bean
public ItemProcessor<Integer, String> compositeItemProcessor() {
CompositeItemProcessor<Integer, String> compositeItemProcessor = new CompositeItemProcessor<>();
compositeItemProcessor.setDelegates(Arrays.asList(itemProcessor1(), itemProcessor2()));
return compositeItemProcessor;
}
@Bean
public Step step() {
return steps.get("step")
.<Integer, String>chunk(5)
.reader(itemReader())
.processor(compositeItemProcessor())
.writer(itemWriter())
.build();
}
@Bean
public Job job() {
return jobs.get("job")
.start(step())
.build();
}
public static void main(String[] args) throws Exception {
ApplicationContext context = new AnnotationConfigApplicationContext(MyJob.class);
JobLauncher jobLauncher = context.getBean(JobLauncher.class);
Job job = context.getBean(Job.class);
jobLauncher.run(job, new JobParameters());
}
}