Java Spring批处理：忽略与指定模式不匹配的任何行

我有一个要求，即我读取具有不同类型输入的文件，如下所示：

*JAMBEG,APP=000007,123456
AC,654321,“ABCD12121212121212”,23423423423424234,ABCDD,23423423423424234,2424,XYZ,ABC,TREX,000000002
AC,654321,“ABCD12121212121213”,23423423423424234,ABCDD,23423423423424234,2424,XYZ,ABC, TREX,000000002
...
AC,654321,“ABCD12121212121214”,23423423423424234,ABCDD,23423423423424234,2424,XYZ,ABC, TREX,000000002
*JAMEND,APP=000007,123456
EOF

我只需要标题行和其后的记录，忽略以TREX、*JAMEND、EOF开头的行

以下是我的测线器的工作原理：

public LineMapper<Customer> lineMapper(){

    DelimitedLineTokenizer lineTokenizerHeader = new DelimitedLineTokenizer();
    lineTokenizerHeader.setNames(new String[]{"association","companyNumber","fileDate"});
    lineTokenizerHeader.setIncludedFields(new int[]{0,1,2});
    lineTokenizerHeader.setStrict(false);

    DelimitedLineTokenizer lineTokenizerBody = new DelimitedLineTokenizer();
    lineTokenizerBody.setNames(new String[]{"type","acNumber","orderNumber"});
    lineTokenizerBody.setIncludedFields(new int[]{0,1,2});
    lineTokenizerBody.setStrict(false);


    HashMap<String, DelimitedLineTokenizer> tokenizers = new HashMap<String, DelimitedLineTokenizer>();
    tokenizers.put("*BEG*", lineTokenizerHeader);
    tokenizers.put("AC*", lineTokenizerBody);

    BeanWrapperFieldSetMapper<Customer> beanWrapperFieldSetMapper = new BeanWrapperFieldSetMapper<Customer>();
    beanWrapperFieldSetMapper.setTargetType(Customer.class);
    beanWrapperFieldSetMapper.setStrict(false);

    HashMap<String, BeanWrapperFieldSetMapper<Customer>> fieldSetMappers = new HashMap<String, BeanWrapperFieldSetMapper<Customer>>();
    fieldSetMappers.put("*BEG*", beanWrapperFieldSetMapper);
    fieldSetMappers.put("AC*", beanWrapperFieldSetMapper);

    PatternMatchingCompositeLineMapper patternMatchingCompositeLineMapper = new PatternMatchingCompositeLineMapper();
    patternMatchingCompositeLineMapper.setTokenizers(tokenizers);
    patternMatchingCompositeLineMapper.setFieldSetMappers(fieldSetMappers);

    return patternMatchingCompositeLineMapper;
}

---

您的意图不是由于错误而跳过对象，而是用逻辑跳过记录；我认为最好的选择是将映射器绑定到

“*”

，并返回自定义对象（如

SkipableRecordBean

），而不是

客户

，然后在

ItemProcessor
中过滤掉不需要的bean一种方法：

让您的
ItemReader
只需读一行并按原样返回即可。因此，读者给出的项目将是一个简单的字符串

编写一个简单的
ItemProcessor
，主要完成
LineMapper
的工作，它基于一个模式，例如：如果项目与一个模式匹配，则将输入字符串转换为您的
客户
返回。如果模式不匹配，只需返回
null
，该项将被跳过

项目处理器的psuedo代码：

class CustomPatternMatchingItemProcessor<String, Customer> 
        implements ItemProcessor<String, Customer> {
    private String pattern;

    public Customer process(String s) {
        if (s matches pattern) {
            construct Customer object base on s
            return customer
        } else {
            return null;
        }
    }
}

类CustomPatternMatchingItemProcessor
实现ItemProcessor{
私有字符串模式；
公共客户流程（字符串s）{
如果（s）与模式匹配{
基于s的客户对象构建
退货客户
}否则{
返回null；
}
}
}


或者更干净：让一个处理器完成从字符串到客户的映射，另一个处理器完成基于正则表达式的字符串验证

只需使用
复合处理器
链接处理器即可。这为每个处理器提供了更好的关注点分离。
我有一个类似的问题，我发送的文件总是有一些不符合正确模式的行。因为我只想忽略这些行，所以我记录它们，并跳过它们

您可以实现
org.springframework.batch.repeat.exception.ExceptionHandler
类似于：

class LocalExceptionHandler implements ExceptionHandler {

    @Override
    public void handleException(RepeatContext rc, Throwable throwable) throws Throwable {
        if (throwable instanceof FlatFileParseException) {
            FlatFileParseException fe = (FlatFileParseException)throwable;
            log.error("!!!! FlatFileParseException, line # is: " + fe.getLineNumber());
            log.error("!!!! FlatFileParseException, input is: " + fe.getInput());
        }
        log.error("!!!! Message : " + throwable.getMessage());
        log.error("!!!! Cause : " + throwable.getCause());      
    }
}

然后将其添加到步骤生成器中：

faultTolerantStepBuilder.exceptionHandler(new LocalExceptionHandler());
faultTolerantStepBuilder.skipLimit(100);

我们开始从LineMapper返回虚拟对象，而不是null，因为null会导致读取器跳过读取其他行。在ItemWriter中，write方法仍然获取这个虚拟对象，您需要在处理它之前进行验证。我们忽略ItemWriter中的虚拟记录

public class FileVerificationSkipper implements SkipPolicy {

    @Autowired
    private Environment environment;

    @Override
    public boolean shouldSkip(Throwable exception, int skipCount) throws SkipLimitExceededException {
        int skipErrorsRecords = Integer.valueOf(environment.getProperty("max.error.record.count"));
        if (exception instanceof FileNotFoundException) {
            return false;
        } else if (exception instanceof FlatFileParseException && (skipErrorsRecords < 0 || skipCount <= (skipErrorsRecords-1))) {
            FlatFileParseException ffpe = (FlatFileParseException) exception;
            StringBuilder errorMessage = new StringBuilder();
            errorMessage.append((skipCount+1)+", An error occured while processing the " + ffpe.getLineNumber() + " record of the file. the faulty record is:\n");
            errorMessage.append(ffpe.getInput() + "\n");
            return true;
        } else {
            return false;
        }
    }
}

faultTolerantStepBuilder.exceptionHandler(new LocalExceptionHandler());
faultTolerantStepBuilder.skipLimit(100);

public class FileVerificationSkipper implements SkipPolicy {

    @Autowired
    private Environment environment;

    @Override
    public boolean shouldSkip(Throwable exception, int skipCount) throws SkipLimitExceededException {
        int skipErrorsRecords = Integer.valueOf(environment.getProperty("max.error.record.count"));
        if (exception instanceof FileNotFoundException) {
            return false;
        } else if (exception instanceof FlatFileParseException && (skipErrorsRecords < 0 || skipCount <= (skipErrorsRecords-1))) {
            FlatFileParseException ffpe = (FlatFileParseException) exception;
            StringBuilder errorMessage = new StringBuilder();
            errorMessage.append((skipCount+1)+", An error occured while processing the " + ffpe.getLineNumber() + " record of the file. the faulty record is:\n");
            errorMessage.append(ffpe.getInput() + "\n");
            return true;
        } else {
            return false;
        }
    }
}

@Bean
public Step step1() throws Exception{
    return stepBuilderFactory.get("step1")
            .<User, User> chunk(50)
            .reader(reader())
            .faultTolerant()
            .skipPolicy(fileVerificationSkipper())
            .processor(processor())
            .writer(writer())
            .build();
}