Spring batch Spring批处理GridTaskPartition不适用于分区器作用域=步骤

我在SpringBatch中有以下配置和代码。我收到了例外

<step id="PROCESS_FILE_TO_STAGING_TABLE_PARALLEL" next="limitDecision" >
<partition handler="partitionHandler" step="filestep" partitioner="filepartitioner" />
</step>

<bean id="partitionHandler"
class="sa.com.mobily.loader.partition.gridgain.GridGainPartitionHandler" />

<bean id="filepartitioner" class="org.springframework.batch.core.partition.support.MultiResourcePartitioner" scope="step" >
<property name="resources" value="#{dataMapprocessFiles}"/>
</bean>

例外情况

2011-11-21 09:56:40,087 458939 ERROR org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:212) Encountered an error executing the step

---

看起来您正试图在步骤上下文之外使用您的partitionHandler，即：

• 创建一个新线程（

  ThreadPoolExecutor

  /Java并发）
• 在新线程中调用

  org.gridgain.grid.util.worker.GridWorker\run（）

• GridWorker

  调用您的

  GridGainPartitionTask

• GridGanPartitionTask

  尝试从Spring上下文中使用

  partitionHandler

这是行不通的<代码>分区处理程序只能在Spring批处理“步骤”上下文中实例化。正确的顺序应为：

• 运行Spring批处理作业（通过作业启动器）
• Spring批处理从上下文获取作业实例，该实例已使用步骤初始化
• 一个步骤将惰性地实例化一个分区器（Spring上下文负责这一点）。请注意，此实例化仅在步骤中有效，例如，步骤实例应在调用堆栈中的前面
• 然后该步骤进行分区，并通过相应的实现将作业分区转移到网格

---

我不理解这一行“一个步骤将延迟实例化一个分区器（Spring上下文负责）。请注意，此实例化仅在一个步骤中有效，例如，步骤实例应该在调用堆栈之前”这意味着您不能

@Autowire

您的partitionHandler，也不能通过

context.getBean（“partitionHandler”）

直接从Spring上下文获取它。它只能连接到

Step

实例，因为它的实例化类似于Step上下文。换一种方式思考：如果分区处理程序中的

#{dataMapprocessFiles}

是在其他地方实例化的，那么它的值应该是什么？请注意，您可能有许多步骤（和许多作业），每个步骤都有自己的设置，并且在运行时有自己的上下文状态（例如，当它们并行运行时）。dma_k：谢谢，#{dataMap[processFiles]}包含一个具有10个文件引用的资源数组。如果您的

#{dataMap[processFiles]}

是常量（也就是说，它引用了您在Spring上下文中其他地方定义的属性），然后从您的partitionHandler bean声明中删除

scope=“step”

，然后高兴起来吧！

public class GridGainPartitionTask extends GridTaskSplitAdapter<PartitionProvider, Collection<StepExecution>> {

@GridLoggerResource
private GridLogger log = null;

@Override
protected Collection<? extends GridJob> split(int gridSize, PartitionProvider stepSplit) throws GridException {

log.info("Executing steps for grid size=" + gridSize);

List<GridJob> jobs = new ArrayList<GridJob>(gridSize);

final String stepName = stepSplit.getStepName();

try {
for (final StepExecution stepExecution : stepSplit.getStepExecutions(gridSize)) {
jobs.add(new GridJobAdapterEx() {
public Serializable execute() {
RemoteStepExecutor stepExecutor = new RemoteStepExecutor("classpath:sa/com/mobily/loader/job/DataLoaderJob.xml", stepName, stepExecution);
log.info("Executing step '" + stepName + "' on this node.");
return stepExecutor.execute();
}
});
}
}
catch (JobExecutionException e) {
throw new GridException("Could not execute split step", e);
}

return jobs;
}

public Collection<StepExecution> reduce(List<GridJobResult> results) throws GridException {
Collection<StepExecution> total = new ArrayList<StepExecution>();
for (GridJobResult res : results) {
StepExecution status = res.getData();
total.add(status);
}
return total;
}

}

public class GridGainPartitionHandler extends TaskExecutorPartitionHandler {

@Autowired
@Qualifier("mscGridGain")
private Grid grid;

public Collection<StepExecution> handle(StepExecutionSplitter stepSplitter, StepExecution stepExecution) throws Exception {
PartitionProvider partitionProvider = new PartitionProvider(stepSplitter, stepExecution);
GridTaskFuture<Collection<StepExecution>> future = grid.execute(GridGainPartitionTask.class, partitionProvider );
return future.get();
}

}

public class RemoteStepExecutor implements Serializable {

private Log logger = LogFactory.getLog(getClass());

private final StepExecution stepExecution;

private final String stepName;

private final String configLocation;

public RemoteStepExecutor(String configLocation, String stepName, StepExecution stepExecution) {
this.configLocation = configLocation;
this.stepName = stepName;
this.stepExecution = stepExecution;
}

public StepExecution execute() {

Step step = (Step) new ClassPathXmlApplicationContext(configLocation).getBean(stepName, Step.class);

logger.info("Spring Version: " + SpringVersion.getVersion());

try {
step.execute(stepExecution);
}
catch (JobInterruptedException e) {
stepExecution.getJobExecution().setStatus(BatchStatus.STOPPING);
throw new UnexpectedJobExecutionException("TODO: this should result in a stop", e);
}

return stepExecution;

}

public String getStepName() {
return stepName;
}

}

2011-11-21 09:56:40,087 458939 ERROR org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:212) Encountered an error executing the step