Java 集成中如何使用executor进行并行处理
我想从FTP读取文件并处理它们。我使用“executorchannel”,但“executorchannel”不会停止接收。然而,当处理速度跟不上时,FTP会继续读取。当文件读得太多时,会出现“过头”异常。如何解决这个问题Java 集成中如何使用executor进行并行处理,java,spring-integration,Java,Spring Integration,我想从FTP读取文件并处理它们。我使用“executorchannel”,但“executorchannel”不会停止接收。然而,当处理速度跟不上时,FTP会继续读取。当文件读得太多时,会出现“过头”异常。如何解决这个问题 public IntegrationFlow processFileFlow() { //dsl return IntegrationFlows .from("ftpFileChannel")
public IntegrationFlow processFileFlow() {
//dsl
return IntegrationFlows
.from("ftpFileChannel")
.enrichHeaders(h -> h.header(MessageHeaders.ERROR_CHANNEL, "exceptionChannel", true))
.transform(transformer())
.transform(new MyTransformer(zipProperties))
.split()
//.channel(MessageChannels.executor(Executors.newFixedThreadPool(3)))
.route("headers['type']",
mapping -> mapping.async(true)
.resolutionRequired(false)
.ignoreSendFailures(true)
.subFlowMapping(DataTypeEnum.DELIVERY_DATA.getCode(),
sf -> sf.channel(MessageChannels.executor(Executors.newFixedThreadPool(2,Executors.privilegedThreadFactory())))
.publishSubscribeChannel(
c -> c.subscribe(s -> s.handle("fileProcess", "processDeliveryData")))
.bridge())
.subFlowMapping(DataTypeEnum.PLACE_DATA.getCode(),
sf -> sf.channel(c -> c.queue(5))
.publishSubscribeChannel(
c -> c.subscribe(s -> s.handle("fileProcess", "processPlaceData")))
.bridge()
))
.aggregate(t -> t
//t.correlationStrategy(c -> c.getHeaders().get("file_remoteFile"))
.outputProcessor(o -> new EventDto(o.getMessages().stream().map(m -> ((EncryptDataDto) m.getPayload()).getEventEntity()).collect(Collectors.toList())))
)
.handle("fileProcess", "endHandler", e -> e.advice(after()))
.get();
}
Spring集成提供了一个被拒绝的执行处理程序 使用具有固定线程池和有界队列的任务执行器,并使用此策略控制并发性。当队列已满时,轮询器线程将阻塞,直到任务完成时空间可用为止