Spring batch 平面文件项读取器，自定义记录分隔符

我需要解析的平面文件

column1|column2|column3$#
data1|data2|data3$#

在哪里

平面文件项读取器：我尝试使用自定义记录分隔符策略，在该策略中，我尝试覆盖

isEndofRecord

和

SuperxRecordSeparator策略setSuffix（）

，但没有成功。它没有将

$#

识别为记录分隔符，因此我收到了一个平面文件解析异常

我有一个解析器univocity解析器来添加自定义的记录分隔符。但是，我不知道如何将CSV解析器设置添加到平面文件读取器方法中

private CsvParser csvParserSetting(BeanListProcessor<Employee> rowProcessor) {
    CsvParserSettings settings = new CsvParserSettings();
    settings.getFormat().setLineSeparator("$#");//$#
    settings.getFormat().setDelimiter("|");//|
    settings.setIgnoreLeadingWhitespaces(true);
    settings.setNumberOfRowsToSkip(1);
    settings.setProcessor(rowProcessor);
    CsvParser parser = new CsvParser(settings);
    return parser;
}

@Bean
@StepScope
public FlatFileItemReader<Employee> myReader() throws FileNotFoundException {BeanListProcessor<Employee> rowProcessor = new BeanListProcessor<Employee>(Employee.class);
    CsvParser parser = csvParserSetting(rowProcessor);
    Request request=MappingUtil.requestMap.get("myRequest");
    InputStream inputStream = awsClient.getInputStreamObject(request.getFileKeyPath());
    CustomRecordSeparatorPolicy customRecordSeparatorPolicy=new CustomRecordSeparatorPolicy();
    //stomRecordSeparatorPolicy.isEndOfRecord(record)
     FlatFileItemReader<Employee> reader = new FlatFileItemReader<>();
    reader.setResource(new InputStreamResource(inputStream));

     reader.setName("filreader");
        reader.setLinesToSkip(1);
       // customRecordSeparatorPolicy.setSuffix("$#");
      //  reader.setRecordSeparatorPolicy(customRecordSeparatorPolicy);

        //reader.setRecordSeparatorPolicy(recordSeparatorPolicy);
        reader.setLineMapper(new DefaultLineMapper<Employee>() {{
          setLineTokenizer(new DelimitedLineTokenizer() {{
            setNames(MyConstats.FIELDS);
            setDelimiter("|");
          }});
          setFieldSetMapper(new BeanWrapperFieldSetMapper<Employee>() {{
            setTargetType(Employee.class);
          }});
        }});
        return reader;
        }

import org.springframework.batch.item.file.separator.RecordSeparatorPolicy;

}

在迭代1中，它正确地解析行标题，当它转到第二次时，它尝试读取行value1 | value2 | value3$#value11 | value22 | value33 而且记录不会被分割以区分一个记录一个记录

最后它失败了

private String applyRecordSeparatorPolicy(String line) throws IOException {
    String record = line;
    while (line != null && !recordSeparatorPolicy.isEndOfRecord(record)) {
        line = this.reader.readLine();
        if (line == null) {
            if (StringUtils.hasText(record)) {
                // A record was partially complete since it hasn't ended but
                // the line is null
                throw new FlatFileParseException("Unexpected end of file before record complete", record, lineCount);
            }
            else {
                // Record has no text but it might still be post processed
                // to something (skipping preProcess since that was already
                // done)
                break;
            }
        }
        else {
            lineCount++;
        }
        record = recordSeparatorPolicy.preProcess(record) + line;
    }
    return recordSeparatorPolicy.postProcess(record);

}

我现在尝试的记录结束方法。 如果header1 | hearers…|$#values | | | | | |在同一行中失败，则此操作将失败。在我的示例中，有126个标题$#values-126$#values-126$#等等

---

我试图覆盖isEndofRecord和SuffixRecordSeparator策略setSuffix（），但没有成功。

你能用你尝试过的内容发布一个失败的测试吗

SuffixRecordSeparatorPolicy

是这种情况下的方法，我不明白为什么不能使用它。非常感谢马哈茂德·本·哈辛（Mahmoud Ben Hassine）的关注。我对问题补充了一些细节。我遇到了一个异常。如果您需要更多信息，请告诉我。您需要在一行上有多个记录吗？如果是这样的话，那么我认为FlatFileItemReader不支持这一点。感谢您检查这一点。是的，不一致。有时，头文件的第一行带有$#，下一行带有值。此文件是由另一个应用程序编写的。如果这些行是逐行的，那么这个后缀可以工作，并且能够读取/解析记录

public static final String DEFAULT_SUFFIX = "$#";
private String suffix = DEFAULT_SUFFIX;
private boolean ignoreWhitespace = false;

 public void setSuffix(String suffix) {
    this.suffix = suffix;
}
public void setIgnoreWhitespace(boolean ignoreWhitespace) {
    this.ignoreWhitespace = ignoreWhitespace;
}
/*@Override
public boolean isEndOfRecord(String record) {
    int fieldCount = record.split("|").length;
   // String recordvalue[] =record.split("\\|");
    if(fieldCount == 126) {
        return true;
    } else {
        return false;
    }
}*/
public boolean isEndOfRecord(String line) {
    if (line == null) {
        return true;
    }
    String trimmed = ignoreWhitespace ? line.trim() : line;
    return trimmed.endsWith(suffix);
}

public String postProcess(String record) {
    if (record==null) {
        return null;
    }
    return record.substring(0, record.lastIndexOf(suffix));
}
@Override
public String preProcess(String record) {
    return record;
}

header1|header2|header3$#
value1|value2|value3$#value11|value22|value33

header1|header2|header3$#value1|value2|value3$#value11|value22|value33

header1|header2|header3$#
value1|value2|value3$#
value11|value22|value33$#

private String applyRecordSeparatorPolicy(String line) throws IOException {
    String record = line;
    while (line != null && !recordSeparatorPolicy.isEndOfRecord(record)) {
        line = this.reader.readLine();
        if (line == null) {
            if (StringUtils.hasText(record)) {
                // A record was partially complete since it hasn't ended but
                // the line is null
                throw new FlatFileParseException("Unexpected end of file before record complete", record, lineCount);
            }
            else {
                // Record has no text but it might still be post processed
                // to something (skipping preProcess since that was already
                // done)
                break;
            }
        }
        else {
            lineCount++;
        }
        record = recordSeparatorPolicy.preProcess(record) + line;
    }
    return recordSeparatorPolicy.postProcess(record);

}

private int getPipeCount(String s){ 
        String tmp = s;
        int index = -1;
        int count = 0;
        while ((index=tmp.indexOf("|"))!=-1) {
        tmp = tmp.substring(index+1);
        count++;
        }
        return count;
    }

     public boolean isEndOfRecord(String line) {
        return getPipeCount(line)==126;
    }