Spring batch Spring批处理分区-分区步骤未执行

Spring batch Spring批处理分区-分区步骤未执行,spring-batch,Spring Batch,我使用批处理分区来根据DB表中某一列的某个范围获取和处理记录。分区步骤stagingPartitionStep在作业配置中的几个步骤之后出现,如下所示: <batch:job id="myBatchJob"> <batch:step id="mainStep"> <batch:tasklet task-executor="myTaskExecutor" transaction-manager="batchTransactionMan

我使用批处理分区来根据DB表中某一列的某个范围获取和处理记录。分区步骤stagingPartitionStep在作业配置中的几个步骤之后出现,如下所示:

    <batch:job id="myBatchJob">
     <batch:step id="mainStep">
        <batch:tasklet task-executor="myTaskExecutor" transaction-manager="batchTransactionManager">
            <batch:chunk reader="myDataReader"
                processor="MyDataProcessor" writer="MyDataWriter" commit-interval="50">
            </batch:chunk>
            <batch:listeners>
                <batch:listener ref="myItemListener" />
            </batch:listeners>
        </batch:tasklet>
        <batch:next on="COMPLETED" to="checkConditionStep" />
    </batch:step>
    <batch:step id="checkConditionStep">
        <batch:tasklet task-executor="checkConditionExecutor"
            transaction-manager="batchTransactionManager" ref="checkConditionTasklet">
        </batch:tasklet>
        <batch:next on="FAILED" to="updateStagingTableStep" />
        <batch:next on="COMPLETED" to="stagingPartitionStep" />
    </batch:step>
    <batch:step id="updateStagingTableStep">
        <batch:tasklet task-executor="checkConditionExecutor"
            transaction-manager="batchTransactionManager" ref="updateStagingTasklet">
        </batch:tasklet>
    </batch:step>   
    <batch:step id="stagingPartitionStep">
        <batch:partition step="processStagingStep" partitioner="stagingProcessPartitioner">
            <batch:handler grid-size="10" task-executor="stagingProcessTaskExecutor" />
        </batch:partition>
    </batch:step>       
<batch:job>

分区器和分区步骤:

<bean id="stagingProcessPartitioner"
        class="com.mycom.batch.partitioner.StagingProcessPartitioner"
        scope="step"> 
</bean>

<batch:step id="processStagingStep">
    <batch:tasklet transaction-manager="batchTransactionManager">
            <batch:chunk reader="stagingProcessorDataReader" writer="stagingProcessorDataWriter"
                commit-interval="50">
            </batch:chunk>
    </batch:tasklet>    
</batch:step>

任务执行者:

<bean id="myTaskExecutor"
    class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
    <property name="corePoolSize" value="20" />
    <property name="maxPoolSize" value="20" />
</bean>

<bean id="stagingProcessTaskExecutor"
    class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
    <property name="corePoolSize" value="10" />
    <property name="maxPoolSize" value="10" />
    <property name="allowCoreThreadTimeOut" value="true" />
</bean> 

<bean id="checkConditionStep"
    class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
    <property name="corePoolSize" value="1" />
    <property name="maxPoolSize" value="1" />
</bean>

分区器实现:分区器为每个分区步骤创建ExecutionContext,并放置一个前缀值,该值将在JDBC查询中用于根据凭证号创建分区

public class StagingProcessPartitioner implements Partitioner{

 public Map<String, ExecutionContext> partition(int gridSize) {
    Map<String, ExecutionContext> partitionMap = new HashMap<String, ExecutionContext>();
    for(int threadId = 0; threadId < gridSize; threadId++){
        ExecutionContext context = new ExecutionContext();
        String stepName = "step" + threadId;
        context.put("voucherSuffix", threadId);
        partitionMap.put(stepName, context);
        LOGGER.info("Created ExecutionContext for partioned step : " + stepName);
    }
    return partitionMap;
}}
公共类StagingProcessPartitioner实现了分区器{
公共地图分区(int gridSize){
Map partitionMap=newhashmap();
对于(int-threadId=0;threadId
数据读取器:步骤上下文中的voucherSuffix在JDBC查询中用于创建数据分区。因此,应在以0、1、2、9结尾的凭证编号上创建10个分区

    <bean id="stagingProcessorDataReader"
    class="org.springframework.batch.item.database.JdbcPagingItemReader" 
    scope="step">
    <property name="dataSource" ref="dataSource" />
    <property name="queryProvider" ref="stagingDataQueryProvider" />
    <property name="parameterValues">
        <map>
            <entry key="department" value="#{jobParameters[department]}" />
            <entry key="joiningDate" value="#{jobParameters[joiningDate]}" />
            <entry key="voucherSuffix" value="#{stepExecutionContext[voucherSuffix]}" />
        </map>
    </property> 
    <property name="pageSize" value="1000" />
    <property name="rowMapper" ref="myDataRowMapper"/>      
</bean>

<bean id="stagingDataQueryProvider"
    class="org.springframework.batch.item.database.support.SqlPagingQueryProviderFactoryBean">
    <property name="dataSource" ref="dataSource" />
    <property name="selectClause" value="SELECT EMP_ID, EMP_NAME, DOB, ADDRESS1, ADDRESS2" />           
    <property name="fromClause" value="EMP_STG_TBL" />
    <property name="whereClause" value="WHERE DEPT_ID=:department AND DOJ=:joiningDate AND VCHR LIKE '%:voucherSuffix'" />
    <property name="sortKeys">
        <map>
            <entry key="EMP_ID" value="ASCENDING"></entry>
        </map>
    </property>
</bean>

问题是,当作业被执行时,每个步骤都会很好地执行,直到分区步骤。partioner创建执行上下文,该上下文可以从日志语句中确认,但步骤ProcessingStaginStep未执行,作业完成时状态为COMPLETED。此作业和分区步骤配置是否正确


下面是日志语句

2015-02-23 03:03:04信息myTaskScheduler-3 SimpleScheduler:146-执行步骤:[checkConditionStep] 2015-02-23 03:03:04信息myTaskScheduler-3 SimpleScheduler:146-执行步骤:[分段步骤] 2015-02-23 03:03:04信息myTaskScheduler-3 StagingProcessPartitioner:29-为已分区的步骤创建了ExecutionContext:step0 2015-02-23 03:03:04信息myTaskScheduler-3 StagingProcessPartitioner:29-为已分区的步骤创建ExecutionContext:step1 2015-02-23 03:03:04信息myTaskScheduler-3 StagingProcessPartitioner:29-为已分区的步骤创建ExecutionContext:step2 2015-02-23 03:03:04信息myTaskScheduler-3 StagingProcessPartitioner:29-为已分区的步骤创建ExecutionContext:step3 2015-02-23 03:03:04信息myTaskScheduler-3 StagingProcessPartitioner:29-为已分区的步骤创建ExecutionContext:step4 2015-02-23 03:03:04信息myTaskScheduler-3 StagingProcessPartitioner:29-为已分区的步骤创建ExecutionContext:step5 2015-02-23 03:03:04信息myTaskScheduler-3 StagingProcessPartitioner:29-为已分区的步骤创建ExecutionContext:step6 2015-02-23 03:03:04信息myTaskScheduler-3 StagingProcessPartitioner:29-为已分区的步骤创建ExecutionContext:step7 2015-02-23 03:03:04信息myTaskScheduler-3 StagingProcessPartitioner:29-为已分区的步骤创建ExecutionContext:step8 2015-02-23 03:03:04信息myTaskScheduler-3 StagingProcessPartitioner:29-为已分区的步骤创建ExecutionContext:step9 2015-02-23 03:03:05信息myTaskScheduler-3 SimpleJobLauncher:136-作业:[FlowJob:[name=myBatchJob]]已完成,具有以下参数:[]和以下状态:[已完成]


是否正在执行子步骤?如果是,每个步骤的结果是什么?另外,分区器中的ID是如何与数据库中的VCHR字段关联的?是的,所有子步骤都正常完成,并且正在调用分区器,我可以看到从分区方法生成的日志语句。因为我需要根据以0,1,2结尾的voucherNumber对记录进行分区,…9我正在传递threadId,它的范围也在0…9之间,将在查询中使用。子步骤是ProcessingStaginStep,所以我很困惑,因为您的问题表明它没有被执行,但您刚才说它是…调用分区器的步骤stagingPartitionStep已执行,但分区步骤ProcessingStaginStep未执行。这是什么作业存储库中每个子步骤执行(processStagingStep的子步骤执行)的结果?