Spring云数据流:JavaDSL只返回最后20个任务执行实例
我正在使用以编程方式在Kubernetes上启动任务。我面临这样一个问题,即该方法返回的值不超过20个。例如,当我启动25个任务时,这个方法返回一个包含20个元素的Spring云数据流:JavaDSL只返回最后20个任务执行实例,java,spring-cloud-dataflow,spring-cloud-task,Java,Spring Cloud Dataflow,Spring Cloud Task,我正在使用以编程方式在Kubernetes上启动任务。我面临这样一个问题,即该方法返回的值不超过20个。例如,当我启动25个任务时,这个方法返回一个包含20个元素的集合,而不是返回一个包含25个元素的集合 代码示例: URI dataFlowUri = URI.create(dataFlowServerUri); DataFlowOperations dataFlowOperations = new DataFlowTemplate(dataFlowUri); Task task = Task.
集合
,而不是返回一个包含25个元素的集合
代码示例:
URI dataFlowUri = URI.create(dataFlowServerUri);
DataFlowOperations dataFlowOperations = new DataFlowTemplate(dataFlowUri);
Task task = Task.builder(dataFlowOperations).name(taskNameWithBatchIdInstanceId).definition(taskDefintion).description(taskDescription).build();
Map<String, String> taskLaunchProperties = new DeploymentPropertiesBuilder()
.put("deployer.task.kubernetes.limits.cpu", cpuLimit)
.put("deployer.task.kubernetes.limits.memory", memoryLimit)
.put("deployer.task.kubernetes.requests.cpu", cpuRequest)
.put("deployer.task.kubernetes.requests.memory", memoryRequest).build();
//Sample code to reproduce the issue :
int maxTasks = 20;
int tasksLaunched = 0;
//Let's launch 25 task instances for our task
for(int i=0;i<25;i++) {
task.launch(taskLaunchProperties, Arrays.asList("index="+i));
tasksLaunched++;
//pendingCount is the difference of tasks launched and tasks completed.
int pendingCount = tasksLaunched - task.executions().stream().filter(
taskExecutionResource -> taskExecutionResource.getTaskExecutionStatus() == TaskExecutionStatus.COMPLETE)
.count();
//wait until pending count is less than maxTasks.
while(pendingCount>=maxTasks) {
//infinite loop if we launch more than 20 tasks (i.e i>20)
Thread.sleep(10000);
//task.exectuions() returns 20 elements even if all 25 tasks completed.
pendingCount = (tasksLaunched - task.executions().stream().filter(
taskExecutionResource -> taskExecutionResource.getTaskExecutionStatus() == TaskExecutionStatus.COMPLETE)
.count());
}
}
uridataflowuri=URI.create(dataFlowServerUri);
DataFlowOperations DataFlowOperations=新的DataFlowTemplate(dataFlowUri);
Task Task=Task.builder(dataFlowOperations).name(taskNameWithBatchIdInstanceId).definition(TaskDefinition).description(taskDescription).build();
Map taskLaunchProperties=new DeploymentPropertiesBuilder()
.put(“deployer.task.kubernetes.limits.cpu”,cpuLimit)
.put(“deployer.task.kubernetes.limits.memory”,memoryLit)
.put(“deployer.task.kubernetes.requests.cpu”,cpuRequest)
.put(“deployer.task.kubernetes.requests.memory”,memoryRequest.build();
//复制问题的示例代码:
int-maxstasks=20;
int tasksLaunched=0;
//让我们为我们的任务启动25个任务实例
对于(int i=0;i taskExecutionResource.gettaskeExecutionStatus()==TaskExecutionStatus.COMPLETE)
.count();
//等待,直到挂起计数小于maxTasks。
while(pendingCount>=maxTasks){
//如果我们启动超过20个任务(即i>20),则无限循环
睡眠(10000);
//task.exectuions()返回20个元素,即使所有25个任务都已完成。
pendingCount=(tasksLaunched-task.executions().stream().filter(
taskExecutionResource->taskExecutionResource.GettaskeExecutionStatus()==TaskExecutionStatus.COMPLETE)
.count());
}
}
上述代码以无限while
循环结束。根本原因是task.executions()
方法似乎只返回任务的最后20个实例,而不是所有25个已执行实例
我检查了数据库中的TASK\u EXECUTION
表,发现所有25项任务都完成得很好(EXIT\u状态为0)。出于某种原因,task.executions()
方法似乎返回20,即使25个任务刚刚完成
如果我更改for
循环,使我总是启动不超过20个任务(例如:for(int I=0;I